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" you will^ I hope^ comprehend and accept as we- 

do the need for thoroughgoing reform in education that 
'President Nixon has called for^ indeed that this Nation is 
calling for. Including of coui;se^ examination and reform ^ 
of the fundamental conceptions underlying the published, 
materials used in the schools.. 

^"What this says to me is that Ve must be prepared to 
question the asefulness of all educational materials pre- 
sently used in the schools^ not goln^ at the review in a 
destructive^ negative fashion too often, the style 

these days but by not being satisfied that we have found 

the ultimate in teaching materials. 

''Much of what is published and used in our schools is, 
of course, excellent and of, undisputed value to the 
educational pi^ceas. But I'm just as certain that many more 

materials are of dubious value, and ^at the ve^y least, lack a 

I 

reasonable basis in research to show,;^hft they are doing the- 
job that needs to be dope. ^ 'We simply have 'to do better, all 
of us. * 

I ' 

S. P. Marland, Jr., Assistant Secretary for Education, 
Department of Health, Educafion,- and Welfare. Taken from ' 
"Educators, Publishers, and Federal^ Policy/' a paper presented 
before the annual meeting of the Association of American 
Publishers, Castle Harbour Inn Motel, Bermuda, Monday, May 1, 
1972. J ^ 



' ^ PREFACE - ' *• . 

, \ In the United States there are' 3 'mil lioir adults who,'for<a variety 
<A reasons ^cannot read.! Another 20 million adults who can read per- 
form at a level so low that they may justifiably^ be described as 
'^functionally, illiterate. 

For many years educators have sought solutions to the problems 
of adult illiteracy. Of particular coqcern has been the identification 
and use of teaching methods and instructional procedures which will 
most effectively teach illiterate adults to read. ' 

Curreot interest in progrAmme^ instruction as a method for 
.teaching reading is. traceable directly to ithe literary works of 
Professor B. F. Skinner in the mid ancj late fifties. An article in 

TI^^E magazine has hailed programmed instruction as the first real 

innovation in teaching since the invention of movable type in the 
15th century." Throughout the country^ many educators believed tliat 
Di/. Skinner had split the pedagogical atom and had found emblazoned 
upon its nucleus the words "programmed learning." We ha4 at last 
discovered a research-based instructional method which^ given time^ 
would do everything from eliminating the tic, in small children to 

s 

teaching illitetate adults to read.' ' ^ - 

. t» 

The distinguished psychologist^ Dr. Lee J. Cronbach^ began in 

A . . - 

^1962 urging educators to demand^test data to (demonstrate the effective- 
ness of the programmed instructional materials they were using. 
There is little reason to believe that educators have taken seriously 
the advice oFj)r. Cronbach and lots of evidence tb indicate they 
haven't. ' * 
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WitTi_a gr'ant .from the ASult Education Branah o*f the U. S... Off ice 
i * - • • 

' of .Education, the Adult Learning Center set out in the summer of 1971 
to conduct a one year research-based fie'ld test of programmed -ivastruc- 
tional materials specifically designed to teach disadvantaged adults 
to read^ The present report presents the findings of this* field test. 

A word of caution t,o the reader is- appropriate. There is nothing 
in the present report which in*^any way validates or invalidates the 
programmed instructional approach to teaching reading. 'The reader is 
urged to remember that a properly constructed experimental tryout^or 
field test of a program may provide an assessment'of that particular 
)rogram, but d^es not afford either proof or disproof ^of (rfie value of 
le general method of programmed instruction. In her very excellent - 
bc\ok, Learning To Read: The Great Debate , Dr. Jeanne S. Chall wrote: 
"Despite thousands '^of research studies and ' scholar ly discussions on 
reading since the turn of the century, it" has been difficult for 
researchers tQ state with any degree of confidence that one particular 
method or approach to beginning reading is ^eally'^better than another. 

The research discussed in 'tTie present report had as its 'single 
purpose the empirical testing for effectiveness of four particul^ar' 
programs designed to teach basic reading skills to marginally literate 
adults. All indications are that this purpose has' been successfully 



achieved. 



/ 



D. Barry Lumsden, Director. 
Adult Learning Center 
November 1972 
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PART I 



PURPOSE, METHOD, AND GENERAL JSfQRMAXION 
PERTINENT TO THE PROJECT 



PURPOSE" \ ■ ' ^ 



The field- of education 'has been deluged with instrugtionai '^*'>i' -^'^l* 
materials that are empirically untested as; far as their learning ' 
effect ivenesT-is concerned. Even In the relatively sophistica^d.^tea *^ • 



of programmed instruction, adequate validating jiata are lax:Xin|- for/' 

f ' 

the large majority of programs: For example^ for only't^n-of a/^am.ple 

of 56 programmed instructional materials fpr teaching Adult.-Basit Educa- 

tion (ABE) students to read do the publishers indicate the availability 

of data, as a criterion'of learning effectiveness '(U. S, Civil Service 

Commission, 1971), Needless to say, publishers' claims .pften fall 

short of standard validation criteria such ^a^'-those sp.ecified by the 

Joint Committee on*Programmed. 'Instruction and Teaching Machines (1966). - 

tjjj^fe' unfortunate' situation^ then^ is "that the consumer of edueal 

tional materials typically mo^t select'^his products on the^basis of 

publishers* cJLaims and of subjective reviews rather than .ott' systematica ' 

eppirically-based evaluations^ As Dr. Kenneth Komoski O^IV) recently 

put it^ "Jhe^ largest single^ group of unprotected consumers in the U. S, ^ 

consists of.milliotis p-f s.tude^nts" who are now delug^ witl;y more than 

/• . > ' 
/ / / — • 

200,000 poorly tested ^textbooks^ films^ teaching. pia'chinesT^d other v 

complex learning gat^ets^' "(p. 33).' Komoski (1971*) .Estimated- that 99 per- 

cent of the na.t:ioi^**^s ,t^aching materials have nevei( beeA systematically 

evaluated on the 6«Lsis^ of amount of students ''.^learning. This lack of- / 

1' /// ^ ^ • ^ ^ ; 

rigorous field tes^ipg.'of instructional materials is not^ hpwever, due 

to our inability tQ j)'^pform tha task;, on the contr^ry^ ^we possess a- 

variety of techniq^ies ^^iEor empirical ^ssessment/^.^, ^ McGuigan and' 

P^ter? (1965). •* ' 



\ t 

' . % .J . \ ■ .... (. 




. ,^»' . ^ materials of questionable ef/ec£^iveness .^;>Th^-_ccui^que^fe^Q,£.,j^nsf fe 
--J-!^_l_^mat^^iai^^^Z^^ £dl tsj^jLj^eTjt and insttetctor^ "Ijutv^he *_ 



lega- 



. -student '-5 <:o ntjir(ued failure'may aXso. i:esai{: in his development of n 
. l.rttjr^ a'tti^t^des' toward^^te^min suclv negative eff^e'ts are even Vs. 

great eJS, ^^^tYi'i AdOilt Basic .Education .student than fc3r the elementary' 
or high school' studeAt^ ^itice tlie adult may make onLy one attempt to_ 



further educate himself.. after droppin^,out of .school. For motivational ^ 
purposes^ the materials ABE students use should produce as much learning^ #^ . 
—4.n as short a time as possible^ *and should also* encourage him to con^^^'v - 
tinue in the ABE program. Since reading proficiehay is necessary for " ^ 
the'XBE studenl:'s f urthe\i^ucation^ priority as far as instructional 
materials are concerned wQul.d hi for* those designed for the effective 

teaching of reading-, . | ' ' - . , . 

. i • , ' -• ' 

• ^ ' , ^ . ' 

It is diffic-ult to underestimate the seriousness and scope of. the *^ " *" 

♦ * / ' 

"reading problem'' among. ABB; students, Those^ who cannot read^ typically ' 

• ^'^^^ * ' ' . * ' 
referP^d to as "functionally illiterate adults/' are usually .unable to J •* 

popui- • , 



» » 

..-•■V 

» f » 
t . % 

< K 
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** f\ ^' f unctloi\ pispductively in our soci^t^. According to the 1960 U. S 

* V lation cen^u^^ about 24^000/000 adult& above the age of 24 years have , 

'\ • r ■ ' . ' r- ■ \ \>.n 

. Ifess than an eighth grade education,. The situation is actually wor.se r * 
\ 



than 



ri't|iis since huU^reds of thousands, of disadvantaged students whor*' f I 
hav^e re^^i^ned in schqp'i^ fof 8/10^ or 12 years cannot read the newapap^-/ ' 
ihdlcatin^^ ^^1^ 8i^^<ies completed is a misleading ipdicatpr pf the ex^tent ; "J ^ 



.of the^probl^ 



/ 



.1 tj 



^' th'e federal government is confronting the problenvof adult _ ^ 
illiteracy with efforts to Cfter traTniTxg-^.uuiler t^^e d^Aended ^Ad^U// / 
E^iacation Act a>f 1366. Still, the^t^rW-enu. remains size^ble/eveii ftho^^lv 

/more than 20 maj6r pieces of legi^laVitJir-i^^support -of adult ^educa^iori"^ 

have been^,/J>a$sed by :th£_^ U. /S. Congress. Fr^m .19^6-1970 only ,2, 25^; 000 : 

' ' ''^ ^^'s^-'v' " . ^ /// /-'^ 

adults jdex the age dfy IS^^^rticipated in ABE programs'^(Nationai Center - 

f or ^ Eduea t iona 1 Stat isel^j^^^ 1^97 0 Hence, iri spite of' efforts by , the ' 

federal gbve'rnmerrt^ still needeid^-^^e "rao^e widespread and effective 
'-' " ' '^^^ ^ — ^ 

^jfoj^^ts so .feha't-mpre thanr ju^t^ ^ small'mlnotrity of ABE students will 

.... • - '■- ''<:■ 

make a sucaeas-f-ul ^e^£orj:,^o^ acquire basic literacy .^kil Is. 

^' ' ' , ' -^-- -^ — * , • .V 

One of^he di£f icalti^T^ith^^V^e^ more effectLtve and wide- 

'AP^^^^^J^T-^^^'^^^S projects is' the frequetrtV^Korta^e through9utv the country 

cff instructional personnel' and* equipment . "'The.*pr\arity in; developing 

metliod$/of Instruction '-should be for tho^e that*'c-an be used -TDV^.the 

learner with relatively little .support 'pBfsonbel.^arid equipment;- -* < 

Programmed^insnfuo^tiohal materials seem to satlstx this requiret^Qnt^ 
' . ' \ ' ' m' y , ' 

In 'addition to the educa.tional advantages frequently pointed qat ^^^^ ^ 
experts in the fields of education and learning It^eory. programmed iriy ' . 
gtruction requires a minimum of teachers* time and.' when used in! book/*" 
form^^ has no hardware requif ements. One possibility/ thei^efore^ is to 
iucri^ase the_ ex cent '-to which programmed instructional material^ are \ 
'useci; Of approximat-eiy' 536^000 j students enrolled in ABE -progrlams A 
spoasorexl by-ihe Adult Education Act of 1966^ 178^821 (33 percent) were .\ 
errrolled in cylasses in' /vhlch' programmed instruction was the ^timary 



educational method employed (National Center'for EduCT^iouai Statistics, 
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In short, the purpose ol this research was to empirically assess 
the effectiveness of avaiiabfe ptcgrammed instructional materials for 
improving ^ttr^lre^^in^ prb'ficiency 0?-f unqtianaXly illiterate adults. • 
In attempting/to'^ec«miiIi%h"thl? i^^QtijvejzJ^^ that a 

program* s effectiveness- varies with .va^x-io^us" chBj;act^ristics of 
students.^ Consequently^ an attempt was made to , assess the effective- 
ness of avail-able^ptograms^^as- -a. function of^^ach student characteristics 
as intelligence, degree of motivation tp sjtudy-, ethnic group, and 
whether or'not English was the studenty-^J^lrst* language. 



• \ REVIEW OF RELATED RESEARCH 

In surveying previous efforts to empiri^ially assess the effective-, 
ness of^rogranuned inWructional materials for the teaclflng of reading 
to functionally illiterate adults^ it became apparent that-only a few 
systematic 'research projects had been conducted; even these resulted in 
limited empirically-based conclusions about program effectiveness.* The 
main function of summarizihg relevant reports here is to emphasize the 

dearth of reliable information in this area., " 

\ 

Expert Opinion (''Sub jective") Evaluations 
While the emphasis will later be on research in which student- 
based objective measures haye been attempted^ there is some value in 
such "subjective" procedures as those developed by Otto and Ford's (1966)^ 
A Check List to Evaluate Adult Reading t^aterials , or Adair's (1968) 
similar instrument^ An Eva luative Iijistrument for Programmed Instruction , 
Such devices can provide profiles of rjeadily observable characteristics'^ 
obtained f f om ^an examination of the material *and manuals. Check lists^ 
as in the case of Adair's instrument^ can provide numeric weight to *the 
presence or absence -o^^^Jected^Ct^^butes and "allow meaningful^ 
numerical underlying comparisof^-J^e^ei^;; any two P.I. courses or 
programs which might otherwise appearH^^ntf^al'' (Adair/p. 2). 
Similarly^ annotated bibliographies, of prograiftmed. ins.tructiona 1 
materials provide some limit'ed information about program characteristics. 
Some examples of these annotated bibliographies of programmed ins^truc- 
tional material are the *Southwe,s tern Cooperative Educational Labora- 
tory's publication^ Adult Basic Education ; An Evaluation of 
Materials (1971); Catalogue o£ Basic Education Systems (U. S. Civil 
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Service Commission, 1971); The Adult: Basic Education -Bi bliograph y 
(1970); A Selected Annotated Mbliograpt^ l^truc.tional Literacy 
Mater ials for ^ Adult Basic Education (1971) 

While these "expert opinion" devices are useful! in assisting the 
instructor to make a decision as to which program, oilt of those 
available, might be appropriate for his students, thejk do not 
provide objective, systematic information regarding prigram 
effectiveness from a learning point of view. Neither d\es the result- 

A' 

ing information have the more general educational value ^ advancing 
researqh procedures for assessing the effectiveness of materials, 

Objective Ass essment of Programmed Materials 
In 1966, Greenleigh Associates reported on a large scale field 

test conducted to determine the effectiveness of four learning systems 

<^ 

to teach economically-dependent adults to "read The students were 
1^15 functionally illiterate individuals, defined as those with reading 
levels below the fifth grade Samples were from New York, New Jersey 

i 

and California The initial reading level was determined by adminis- 
tering thel^Gray Oral Reading Paragraphs The students studied two and 
one-half hqurs per day for 17 weeks in one of four learning systems, 
identified as "Learning to Read and Spell" (America Incentive to Read), 
**Reading in High Gear" (Science Research Associates), "The Mott Basic 
Language Skills Program" (Allied Education Council), and "Systems for 
Success" (Follet Publishing Company) The researchers attempted to 
form control^ groups but were unable to acquire a sufficient number of 
adults to comprise such a condition. 'The primary d^ta were presented 
as mean gain scores (mean posttest scores minus mean pretest scores) 



* 

r 
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in reading achievement, computed on the basis of The Gray Oral Reading 

I 

Paragraphs and by The Iowa Test of Basic Skills.' 3^ent attrition 
was 46 percent. The gains Were tested for significant difference from 
^ 0^ and analysis of variance was used to determine whether there were 
*; significant differences in student gain scores by reading systems. 
The -primary conclusion was that "almost all students showed some gain 
^ . in reading ability daring thj field test. However, there was no 
significant difference in .student? gaia scores by reading systems." 
More particularly^ they failed to find significant gains by The Gray 

Oral, Reading Paragraphs^ but did find some on the Reading Comprehen- 

^'t _ 

sion^ Work Study Skills^ Arithmetic, and Composite scales of The Iowa 
Test of^ Basic Skills. While^^the most positive results were statis- 

% 

tically significant^ gains a^gpeared to be small and^ of doubtful 
practical significance. Andy of course, an analysis to determine 
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thaj: a given gain is or is not statistically significant doesn*t tell 
one anything about the^size;^of chat gain score. 

Griffith (1967) rather|/vigorously' evaluated the Greenleigh report, 
one of his criticisms being that students and teachers were not 
assigned 'to learning systems at random. Since the Greenleigh research- 
ers took as a ma j^or purpose that of comparing effectiveness of the 
four systems, G?:JLffith pointed out a most important confound. ^ — 
Consequently, the meaningful comparison of gain scores among the fbtiyf, 
systems was precluded, Griffith also pointed out that a number of 
"findings" were but casua;! observations or subjective generalizations, 
and the indication is that the main value of the Greenleigh report 
was to show the large number of variables that must be de^lt with in 

] ■ . , ,. 



ABE field testing researchv Thus, zhe Greenleigh report dlearly > • 
established the difficulty of aJt:quateiy dealing with methodological j 
problems in this area. 

In a relatively minor evaluation study of a reading system by 
Heding, Ames^ Artley, Grimsley and Andrews (1967); 18 (53 percent) 
of 34 adults who Were illiterate remained for the conclusion of the 

.experim^t after Some 90 hours of ^literacy instruction.' Only small 

J*' 

and nonsignificant mean gain scores occurred for measures of word 
meaning, paragraph meaning and word study skills. There was^ however, 
^ significant increase on a measure of spelling. 

Steuart (1968) conducted a study using 86 Spanish-speaking 
sijj^nts who were functionally illiterate in English. They studied 
the Mott Basic Language Skills program and the Sullivan Programmed 
Reading for Adults. Steuart states that he was unable to establish a 
f unction^lli^ illiterate control group within the confines of the 
Mexican-American population. The .data indicated that achievement was 
significantly higher on the .reading compre^ejnsion posttest for the 

r> ' ' ' ~ 

students who studied the Sullivan program than for their Mott treatment 
counterparts^ but there was no difference in mean gain between the two 
groups on measures of word recognJ.tipn^ vocabulary^ and spelling. The " 
reading, vocabulary^ and spelling measures were scales of the Adult 
Basic Learning Examination (1968). 

While there are several other indirectly related studies' rn 
this area, the ^general conclusion seems apparent that 
there a#e few satisfactory data with regard to assessment of*Jhe 
effectiveness of programmed instructional packages' for the teaching of 
reading to functionally illiterate ^dult'sj 
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METHOD ■ ^ • 0 ■ 



Students 



In an attempt to ide^ntify functionally Illiterate students to be 
used in this p^oject^ 1^163 candidates throughout the United States 
were administered the Reading Comprehension Section Scale of the" Adult 
Basic Learning Examination (ABLE), Level 2, Form A. Students with raw 
scores^ from ,10 through 53 on the ABLE were admitted^into the program 
(icf. , "Assignment of Students to Programs/' p. 20 ) . of the 1,163 
candidates tested (see page 13), 531 started ind .completed^^he course 
vpf study, as, defined on p. 32ff. The number of these students in each . 
field test site is specified in ^able 1, along wi^th the dates ^t^^yhich 
the project was started^.and completed at each site. 'As can be 'seen, 'an 
effort wa's n^ade to sample a number of ethnic groups and all=regions of 
the country ^(north, east, west, and south). ^ ^ 

Tables ,2 through 6 present estimates of studenjt characteristics 
for the total sample and for those at each' site.* Thus, it can be 
observed that there were approximately. 303 male and 221 female students 
in the total sample (Table 2) To preserve anonymity of the test 

• ^ ■ V - - 

sites^ as promised those cooperating with us^ site numbers ib" Tables 
2«6 do not correspond with the order of those' in Table 1. The 
numbers of students in various age categories *^fe' presented • in Table 3 
wher6 it can be seen that the modal age was 18 years (120 students), <» 



We will soon turn to the problem of missing data, traditionally 

a most s^^rious problem* in Adult Basic Education research. Missing 

data on s^onje student characteristics in Tables 2'throug1i 6 explain why 

the totals^ do not always equal the number of students who actually 

completed the project (viz. 531). 

I 



and that the second most frequent ag^. category was 25 years 
(approximately 92 students). The data in Table 4 show that the 
largest number of students was black (about 271) and the second most 
^frequent category was white (113), The first language spoken 
by approximately 413 students was English, ' whereas English was the 
second language for approximately 93 others (Table 5). The eighth 
grade was the modal grade completed (176 students) with the second ^' 
most frequently .completed grade bein^ the ninth (93 students)^ 
according to the data presented in Table 6, 

Selection of Programmed Instructional Reading Materials 

The criteria for the selection of the instructional systems were 
as follows: 

_ 1. Only programmed instructional materials were considered* 
TRis meant that the materials employed self- instructional' procedures 
so that the students cQjild learn the subject matter with little or no 
outside h^ip^ Furthermore, the materials incorporated a prearranged 
systera requiring the learner co respond actively for each unit (frame) 
of materialo ' Additionally^ the materials provided prompt confTtmatio'n^ 

rection for each respanse the learner made (cf Joint Committee 
on programmed Instruction ^nd Teaching Machines, 1966). 

2 The programmed materials were in software form, requiring no 
hardware as supporting equipment (as 'mentioned on p. 5 ), 

3. The publisher indicated chat the programmed materials were 
appropriate for an adult population. 

4. The materials w^re applicable to thev grade leyels that define 
the functionally illiterate adult, viz ^ > below the eighth grade level 
and preferably starting w^ll below that grade level. 
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Table 1. Field Test Sit^s, Operating Dates, and Numbers of Students 



Site 



Adult Learning> Center 
Raleigh^ C 

Correctional Center 
for Wcmen, Raleigh, 
C. 

Caledonia Correctional 
Center, Jillery, N. C. 

npower Development 
aining Prcgram, 
reenviUe, N. C. 

Adult Basic Education 
Center, Lcmberton, 
C. 

Learning Laboratory, ' 
Cha i: tanooga ^ Tenn. 

Centinela Vallev Adult 
School, Inglewood, 
Calif, 

La Puente Valley'Adult 
School, City cf 
Industry. Calif. 

Meridian Junior College 
Meridian, Mass. 

Itawamba Junior College 
^Tupelo ^ Miss« 

Knoxville Manpower 
Training Center, 
Knoxville, Tenn, 

Beaumont School for • 
Boys^ Beaumont^ Va. 

State Industrial Farm 
for Women, Goochland^ 
Va. 

Onward^Vith Learning 
Center (OWL) . 
Springfield . Mass^ 

Men's Prison ^ 
State FarrT', j^a,. 



Scart 



End 



Students Students 
Tested Completecl 



10^26-71 . 5-28-72 36 



10«27-7l * 12- 9-71 ^ 40 



10-27^-71 12- 1-71 • 44 



12- 9-71 ' 3- 8-72 32 



2- 7-72 5-18-72 48 



11- 1-71 4-17-72 40 
VI- 9-71 '4^28-72 > 150 

11-10-71 5-12-72 ibO 



10-10-^71 4-28-72 110 

10- 8-71 3- 1-72 75 
1^7-72 5-19-72 ' -~50. 

11- 2^-71 5-10-72 150 
11-15-71 3- 1-72 83 

2- 1-72 ' 5-12-.72 90 
♦ 

11-29-71 5-15-72 115 



22 

.20 

' 18 

10 

13 

16 
64 

60 

51 

23 

16 



88 

60 

23 



Total 



1163 



531 
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Table 2* Frequency of Gencier of StuTdents by Site 



Site Number ^Male Ferjale ^ Total 



1^ 



1 I . - 4 18 '22 

2 ,' 9 . ' * 41 * 50 



« 
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3. - 60 0 * 60 •• .i 

.4 • ^0 '20 " 20^ * 

• 5. . 7 _ its ' 23 

6 '-IS' -0 N ,1^" 

7 ' 37 ' 23 -60 

8 ■ Q ■ . 23 ^ 23 

.9 29 34 . 63 ^ ■ 

IjO- ■ .6 7 ' * • ■ 13 ■ 

11 • 10 ■ 0 ' 10 



12 - 27 20 47 

.13 '-1 ' 15 U 



14 , ,V — V-13-- - " 3 .16 

15 • .82 -1 83 

1 

Total , 303 • " 221 ' 524 
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5. Frequency of Primary^ language Spoken in Home by Site 



Site 
Number 



English 



Other' 



1 




1 s 


2 






3 




59 


4 




20 




b 




5 




ZZ 






1 ft 


7 




. 32 


8 




23 








10 




r 


^ 11 




/ 10 


12 




/ 28 








13 




/ 


14 




15 


.15 




76 



7 
0 
.0 
0 
0 
0 

V' 

44- 

0 

0 
16 

0 

. 0 • 

0 



Total 



22 
46 
59 
20 
22 
18 
58 
. 23 
^4 
13 
V 10 
' -44 
16 
15 
76 



Total 



413 



93 



506 
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J An extensive search based on some 20 bibliographies^ letters to 
publishers^ and advice of consultants indicated that four appropriate 
programs were commercially, av'ailable. .They were: 

1. -Building Your Ung^age Power , by Frank C. Laubach^ ^ro^rammed 
by William Ct^ Wolf Jr. , published by Silver Burdett Company (1965). 

/ 2- Progranmied Reading for Adults , by Cynthia Dee Buchanan^ 
published by McGraw-Hill Company ' (1967) • 

3'. Bgilding Reading Power , by Joseph Loretan and ?kelly Umans^ 

• ^ . * ' /J ' 

published by Charles E, Merrill Books^ Inc, (1964).' 

/ i * ' * ' 

4. ^ Reading Series , by Maurice W. Sullivarf^ pCiblished byV 
Behavioral Research .Laboratories (1969). - ' - 

After the. selection lof the above programs^ the ^publishers were 
again contacted with a request for the entire program^ for a statement 
about the student population for which the program was intended, 'for 
any tests available for use with their program^ and for any validating 
data available for the program. These programs were then used in the 
field test according to instructions specified by , the pablifeKersl No 
validating data for adult populations were received f rom "-^tiP^QlE" t:% 
publishers. Characteristics jof'^^hese programs are specified .under " /' 
"Results" and appear in Appendix A. ' . ' 

V - ■ 

selection of Field Test" Sites 

Each of the 50 State Directors of AduU Education was contacted 

to determine if his state might be interested in cooperating in this \ 

research project. Those who indicated art interest were contacted agailh 

and final sites were selected according, to the wi/llingness and the \ 

* ' i ^ ' ' ' ' 

ability of the site personnel to provide a* oreasonable number of 
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students. An effort was also made to selecr sites that ha'd\students 

wich varied ethnic characteristics and ro obLaln some degi'ee of. ) 

national sampling (north, sottth^' east, arid west) o In addition^-' 

because of the typically high attrition among ABE students^ two^ 

prison sites were established near our administrative offices in 

North Carolina and three in neighboring Virginia. As a result^, a 

^ ^ • * 

, total of 15 sites was established in six stated;, five were in North 

\ * 

* . Carolina, two in California, one in Massachusetts, two in Tennessee, 
^^^hj^ee in Virginia^ and two in Mississippi (see Table 1). 

* \ Assignment of Students to Programs 

fhe Adult Basic Learning Examination (ABIE) limits for each .program 
group established by extending two standard deviations above and 

vy*. below the ^rad*:^ levels that were specified by the publisher for each 

'>.-. ■ 

\program. Thia,c extens ion allowed an enlargement of the student sample. 
IC\^hould be recognized that there Is a lack of comparability of grade 
level^*$mong ABE materials so that* a given grade level does not mean 

V r 

tK'e^^me^hing fat different reading programs or for different 
standa^i^^^\^es ts. ^Furthermore, as is commonly recognized^ the 
concept oX'^^^s^^level" is inappropriate for adults. The resulting 
Adult Basic Le^ning*.^xaminaci9n limits according to which students 
^ were assigned to. p^ogram^a re presented in Table 7. 

A student rost^^r was d^^loped in which the order of the four 
programs was rotateq^.^ that thK»first student at a given site would 
be assigned to theXfl^r^t program, th^. second to the second program^ 

•i 

the third to the thi#rd jVrogram^, the £o^rth on the list- to the fourth 
fjrogram^ the fifth on tAi^- list ko the first -program, and., so forth. 



labje 7. Range of Grade Levels and Limits of Adult Basic Learning 
Examination Raw Scores on the Reading .Scale 



Grade Range Readability Level Raw Scores 

(from Publishers' '(from Civil Service on ABLE 
Catalogs) " Commission^ 1971) Level 2 



Programmed Up to 6th 1.5-6.0 ' 35-50 

Reading for Grade Level 

Adults y 



Reading Series 0 to 8th o.G - 8.0 10-50 

Building Your Not Listed 0.0 - 4.1 26-44 
Language Power 

Building 4:5 or Higher , 5.0-7.0 38-53 
Readings Power 
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This procedure obvicusly wouTd have resulted in the random assignment 
V of stadencs to all foar/condxcions . however., it can be 3een in 
Ta'ble 7 that che programs nave differing ranges of grade levels; 
consequently, when a student had an Adult Basic Learning Examination 
^reading score that was too low or too high for che program assigned to 
him^ he was reassigned to the program that was necessarily appropriate 
for.him* His first program was then assigned to the next available 
student on the list. Students who, did not score within any of the 
program ranges were excluded from the reading project This restric- 
tion on ability to randomly ass ign. students to che four program 

condicions necessarily introduced a confound in the experiment. 

t 

Furthermore, it later turned ^ixt that some sites v/ere unable or 
unwilling to use all four programs concurrently. Limiting reasons 
were practi'cal problems; some proctors could not handle four dif- 
ferent procedures for the different programs; some sites had so few 
students that' it w^s impractical to incorporate four different 
reading pi:ograms : at a number of sites fev or no students scored on 
the ABLE in the range of the high and/or low programs (Table. 7)^ so 
that all four programs could not be assigned at some sites. Such 
constraints thus dictated that students with differing initial 
reading levels be' assigned to the four prc^rams^ and sometimes to 

* m 

fewer than fear programs at a given site, ' The programs used at each - 
site are specified under "Resulns'\ • . ' ^ 

• Since It was not possible-tc randomly assign ail students to the 
fout program conditions at ^any site, the potential for meaningfully 
comparing the effectiveness of the four programs became limited. Even 
so^' it is more desirable to judge a program by a more absolute criterion, 

^. — ^ . . 

60 



^ • . 23 

For instance, it doesn't .help much to be ab\e to say that one progl 
is better than another, if' all four are poor. The goal, therefore, was 
to determine the effectiveness of each program when judged in isolation 
from the others. To accomplish this, as explained later in this report, 
the strategy followed involved the ^sseysment of each program^s effec- 
tiveness relative to a large sample of progr.ams which had already been 
evaluated (cf . , McGuigan and Peters, 1965). 

> Use of Programs 

Air programs were used as specified in the accompanying publisher- 
produced manuals or teachers* guides. The class situation, study 
schedule, etc., varied considerably among sites, depending on local 
^ demands and p^rocedures."^. In some, the student worked at times of his 
own choosing, while in others, all or only parts of a class worked on 
their reading programs for the projec^. The number of meetings per 
week ranged from 1 to 5, with the majority of the students working 
two or three times a week,; typically for about an hour per meeting. 
No student studied material designed to teach reading, other ^han the 
assigned program^ though 6f cdurse some did study other materials such 
as arithmetic, English and vocational subjects.. 

' Reading Series employs a series of booklets to be read and 
• • . ■* 

coordinated with th^ books of the program; these were used as 
specified in the manual. None of the other programs* had correlated 
or supplementarjriiiaterials. that were specified as integral parts of 
the progtafir. 

The students who "studied Building Your Language -Pt) we r and 
Building Reading Power all started in the first book. For the other 
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two programs the students started in the book appropriate for them as 
indicated by the placement^ examinations . ^ 

Mea sures Taken 

Proctors were instructed in detail with regard to data collection 
procedures, including administration of all tests, the safeguarding of 
the data, etc. Each site was clos6ly supervised by one member of the ^ 
project staffl Step-by-step procedures for the proctors during the 
data collection and study phases are specified in Appendix The 
measures taken were as follows: 

1. The Standard Progressive Matrices , a nonverbal measure of 
intellectual capacity developed by Ravfen (1960), was administered to 
each student as an tintimed "capacity" test at a convenient time during 
the project, 

2. • The» Adult Basic Learning Examination (ABLE) ^ a power test 
developed by Karlsen^ Madden and Gardner (1967), was- administered as 
a screening device, and also as a pre- and posttest for a measure of 
learning. The students were administered, the vocabulary, reading and 
spelling scales of Form A. ABLE Level 2 ranges from below grade 3 to 
grade 9+, While it obviously would have been desirable to use alter- 
nate forms of the Adult Basic Learning ^Examination,' Boyce's (1970) 
wor^ raised considerable question about the comparability not only of 
the alternate forms of the Adult Basic Learning Examination but of the 
t^o different levels. Other appropriate standardized tests were^ con-^ 
sidered no better in this respect. 

T. A Content Test is a eurricuiinn-embedded itis^irument that , 
samples the, specif ic material presented in each book. of a given 
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program. Conte'nc Tests were administered as pretests' and posttest^ 
• immediately^ Wore and afner studying each book. There were 20 
Content Tests for Reading Series , six for Programmed Reading for 
Adults, 15 ^dr Building Reading Power, and four for Building Your 
Langua ge Power . Tests furnished by the publisher were used as 
Coateoj/'^sts in the case of programmed Reading for Adults and 
Reading Series , but had to be developed by project personnel for the' 
programs Building Your Language Power and Building Reading Power (s.ee 
Appendix C for reliability estimates of the latter two Chat were staff 
developed). The Content Tests used for Reading Series . were "Progress 
Tests"; as advised by the publisher^ half of each was administered 
as a posttest when the student completed h'alf of each book and the 
remainder upon completion of the entire book^ Both halves were ' 
administered together for a pretest. The Content Te3ts for Programmed 
Reading for Adults were ''in-book'' tests which occai^\ach 24 pages in 
tlie program booklets and evaluate the student's achievement* on' the 
preceding 24 pages. 

For Building Reading Power, application exercise^ furnished by 
the publisher were used* as the basis fop^the Conteht Tests, where they 
were available, ' ' " * . 

4. General Reading Test. The* General Reading Test ^as developed 
as an additional precest-pos ttest measure of learning, and also, to 

study the relative performance for each program baked on a sample of >Vr 

I \ 

measures of other programs.* This test consisted of 40 items thatLere 
randomly. selected from the Content Tests o£ the four programs. ^ / 



Special thanks to Dr. Edwin Smith of Florida State University and 
to Dr. William S. Griffith of the University of Chicago for this suggest 
tion. . • * 
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" / 5. Individual Reading Placement Inventory (IRPI)^ (Smith and 
WejLdon^ 1969). The Oral Paragraph Reading section of this test was 
administered as a pre- and posttest to a small sample of the students. 
For this^ proctors at four sites were famished with tape recorders^ . 
and tapes^ and instructed in the administration 'of Form A of this 
test (see Appendix D) 

6. Slosson Oral Reading Test (Slosson^ 1963) , The 'siosson Oral- 
Reading Test (SORT) is also an.andividual reading test. However^ in 
this the student pronounces words at increasing levels of difficulty. 
Administration of the Slosson Oral Reading Test was performed with the 
same students as for the Individual Reading, Placement Inventory (see 
Appendix D) . . 

7. Program Placemient Examinations . Reading Series and Programmed 
Reading for Adults include placement examinations that indicate where 
in the pro-am (j^. e. ^ at which book number) the student should begin ' - 
his study.' These placement tests were administered ag pre-' and post- 

A 

tests^/ . ' 

8. Student Motivation Scale . At the conclusion^ of ^11 testing^ 
the 'proctors were asked to rate each student's level of motivation whil 
studying his program. A three-point , scale (high^ medium^ and low ^ 
motivation) was furnished each instructor for all students. 

9. Initial Interview Form . This form was completed by the 
proeter^-GTr the^asis of existing student records and on orally fur- 
nished information gained froni the student prior to his start in th& 
reading program. 



6'/ 



27 _ 
10. Student Time Log . Under proctor supervision^ students 
logged their study time and the material covered during eacjh day of 
their participation in the program. * 
^ ^ 11. Student Evaluation Form. Student evaluation forms' developed 
by MaGuigan and Peters (1965) were slightly modified for the adult 
population. The details of this questionnaire are presented in the 
"Results" section. 

12. Proctor Evaluation Form. On termination of each site the 
proctors completed a standard evaluation form developed by McGuigan 
(1971) for each of the jHfOgrams used ab^'that site. (See "Results", 
Tables 54-60.) ^ ^ 

Method of Measuring Learning 

r 

■ Amount learned is the primary criterion of the, effectiveness ot 
each program (McGuigan. and Peters, 1965). The amount of learning 
resulting from the use of each program was assessed by means of the' G 
statistic, the ratio between the amount learned (gain) to amount that 
could have been learne^ (possible gain) as measured by the tests. The 
equation for computing G is: 

CI 

Q ^ (mean posttest score) •> (mean pretest score) 
— (possible ''score) - (mean pretest score) 

To compute G we determine: 

1) the possible test score, the actual number of points or items 
on the test. 

2) the mean pretest score for the group of students (by averag- 
ing the number correct for each student on the pretest). 

• 3) the m^an posttest score (the mean number of correct items by 
the group of students after completion of a given amount of 
material) . 
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4) the amount .learned (gain score) by subtracting the mean 
jfretfest score from the mean posttes't score^ the numerator • 
9f the equation^lbr G. The percentage gain is the gain score 
divided by the mean pretest score (multiplied by 100), 

5) the amount of possible learning (the denominator of the G 
* ratio) as measured by a Lest, This is the possible gain 

* score^ viz the possible score minus the mean pretest score, 

G. thus^ is a value that normally ranges between 0 and 1 such 

it 

that the higher the value^ the greater the amount learned. (A nega- 
tive G^ incidentally^ results when the posttest score ^ less than 
the pretest score, probably due tOx chance fluctuations ,).* Furthermore^ 
a G of .'50 'or higher indicates a satisfactory amount of learning 
(McGuigan and Peters, L965), A distribution qf empirically derived G 
scores for programmed and nonprogrammed classes is presented in 
Figure 1. • * 

Among the advantages of G over more primitive measures pf amount 
learned^ such as a raw gain score^ is that it takes into account the 
student *s level of proficiency prior to his stuiy of the learning 
system under test. Consider a student who has a very low initial 
level of proficiency such as that indicated by a pretest score of 
25 percentj he has tile probability of showing a' much larger gain 
percentage than a atudent whose pretest score is 90 percent. Hence^ by 

^ 

using a simple gain percentage as an index of amount of the learning 
effectiveness of a program/ a negative evaluation of th6 program 
could result if student^ with high protest scpres were used^ even if 
the program were effective . With the G statistic^ however^ a program 
may be judged to be effective even with high pretest scotes 
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Fig. 1. Frequency distribution of gain-to-possible-galn ratios (G) for 24 different programs. 
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since the index of effectiveness is amount learned relative to amount 



^3 •fly 

that could be learned, -^The specific applications of G to the 
various dependent variable measures will be specified under "ReBuItsJ^ 



V 



*Note that the higher the pretlest score^ the higher tHe value 
viz^^ the empirical correlation (reported by McGuigan and Peters, 
1965) is .14 (P'< .0I)o ' 



31 



THE PROBLEM OF MISSING DATA AND ATTRITION 

According to. the National Center for Educational Statistics ^ 
(1971), of about 485,000" adults who participated in Adult Basic 
Education programs during the fiscal year 1969, approximately *172, 000 
(35 percent) separated during the year for reasons other than having 
completed the particular program in which they were participating. 
In a' research project, greater demands are placed on the student 
and, consequently, the attrition can be expected to be even greate'r 
than 35 percent. I'he following are some reasons for attrition: 

sickness,, gaining employment, change in family situations, change in 

> 

residence, and lack of interest. In .addition, tes'ting is relatively 
frustrajting for the. functionally illiterate adult, and research 
testiiig demands, ^are quite high. Consequently, some students avoid 
the testing procedure when possible. In^e Gree^aeigh (1966) and 
Heding et al. (19^7) studies cited earlier, ,the ^tt*ition rates were 
46 percent and 53 percent. Since attrition is such. a major problem 
in ABE studies, in order to maximize the quantity and quality of d^ta 
coU.ected, and to reduce. student attrition', 'the plan for this project 
was to as fle:tible as poss^ibJLe with regard toj^he length of time 
that the students worked in their programs. Consequently, while every 
effort was made to keep the studelrfe^^i^^ing to complet^thelr -entire 
ptograms, it was possible' for them to. Ite^ posttested*^ (and included in 
the data analysis) at^any^ /time af ter^tfre-^completion of one book. Only 
16 students failed to complete the first book in their programs; the 
attrition rate in this project can thus be considered to be inconse- 
quential. Of the. 1,163 candidates who were tested, the'^Adult Basic 
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Learning Examination Scores for &1'6 did , not fall within the limits 
^specified in Table 7. The remaind^ J531) were the students for this 
project, - ^ ^w.— ^""^^ 

The relevance^oJ^ attrition is^ of^course^ important to any 
question'^p.f ';generaliz^fion to^^ population sampled. ^However^ such a 
^uiBstion^ts-' far too advanced for the present state of ABE methodology 
/aitd. SO ia^uot especially germane here. This is to say that^ even if 
Iwe could adequately define^ describe^ and* enumerate a population of 

functionally illiterate adults, it would be next to impossible to 

' ' ' * 

obtain a representative sample short. of dictatorial powers* Fgr such 
reasons^ with but minor exceptions, , techniques for data analysis are 
Umited to those of descriptive statistics in c^ont-ras.t to those of 
inferential statistics. ' • 

In this respect, see ^61s6 ,Tloyd (1972). ' 



PART II 

LEARNING RESULTS FOR PROGRAMMED READING FOR ADULTS 



♦ 

results' AND DISCUSSION 
Learning Data 



, As previously stated^ tne learning criterion is by far^the most 
important for assessing the ef fecr.lveness of a program, A^er 
analyzing the learning results for each program, «the- remaining' 
criteria will be discussed. 

Learning Results for Programmed Reading for Adults 

Content Tests , Programmed Reading for Adults was used at all 
15 sites •specified in Table 1, In Table 8 are the learning results 
for each book^ based on the content tests. It can be seen^ for 
instahte^ '^hat ^27 individuals who studied 'Book 3 had a mean pretest - 
score of 83 percent with a range of means over sites from 69 percent 
to 91 percent. Their mean posttest score was 91 percent (range 85 
percent to 100 percent)^ yielding a mean gain- of 8 percent^ and a 
percentage; gain of 10 percent ^(see pages 27 to 30 for the method 
of computing these statistics). The mean G ratio of ,46 was signifi- 
cantly different from zero^ and the range of G v^lMes ov^r sites was 
,34 to 1.00. The G ratio's for Books 3 and 4 approach the^criterion' 
of \50^ suggesW-ng that the individuals who studied this program may 
(within sampling fluctuations) have learned a satisfactory amount from 
these particular books.- The amount learned from the other four books 
seems^ by the content test measure, rather low^ viz, G values of .30^ 
.07^ ,25 and .28. • ' 
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I .. 

It may oe, noted thafc the students generally had quite high pretest 

l . I 

Scores— based on 329 scores, sannned at the bottom of Table 8- the mean 

pretest score over all six books was 91. The mean posttest score over 
bobks was 93, the gain was 2, the percentage gain shown in parentheses 
was 2 percent, and the overall G was .22. 

The results in Table 9 present the G and gain percentages as* a 
function of intelligence level. More particularly, students were 
classified according to whether their scores fell in Raven's (1960, 
p. 11) category II ( "def ia^itely above the average in intellectual 
capacity"— above the 75th percentile) , category III ("intellectually 
average," a score between the 25th and 75th percentiles)', or category 

IV ("definitely below average in intellectual capacity," i.e , below 

t ' 

the 25th percentile). It may bs noted, for instance^ that 10 students 

with Standard Progressive Matrices scores which placed them in the low 

intelligence category had a percentage gain of 14 percent and a G 

value of .45. Their G ^at^-s for Bocks 4, 5, 6, 7, and 8 were .18, 

. 25, .09, .23. and .^39, respectively. These G values for individual 

books were weighted by the correspondflfeun'ioer of students (n) 

summed and divided by the total number of students (N) to provide a 

mean' weighted G over books. That is, a composite G was computed 

according to the eqjuatioti: / 

^ i 

In this case; * ^ 

4 * / 

I 

- ' Composite G- =- - ^^(^O) ^-18(14) 25(26) ^.1^2^) ^.23(27) -.39(31» 

' - • . / ^ - 

. 33.98 ' 

132 \ 
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This composite G for the low intelligenc-fe individuals of .26 i^ 
presented ^n Table 9,, as are those for -pSe medium and Mgh intslli-' ' 
gence categories (viz., .38 and .65). One should cautiously inter- • 
pret these ,resulcs, though, since the number falling in the high 
intelligence category is relatively small, ' . 

The results for this sample 'Sugg est that intelligence (as-f,. 
measured by the Standard Progressive' Matrices) is positively related 
to amount learned through the study of Prograiimied Reading for Adults . 
More particularly, while those of low and medium intelligence did not 
appear .to leam^a substantial amount^ chose cla£§ifie4 in the high 
intelligence category apparently profited substantially from^' studying 
■ the program. ^ ' - - 

As with intelligence, motivation of the student.s seemed to be , ' 
positively related to the. amount learned (Table 10). Even so, while 
i the highly motivated individuals appear to have learned more * 

(composite G = .41) than the individuals with low m9tivation (composite 
G = .09) even those classified as high in mot^at ion Wy approached 
•the criterion that indicates a sati-sfactory aciptint of- learning, viz., ' 
G = .50. With regard to individual books, it is fntyerssting - to 

\ > " •> ° O , ■ V ^ 

note in .Table 10 that three ^ the six books were successful for 
Jt-. • 

•the highly motivated students, while only two of 12 'possible books ' 
were successful for the other students (viz.' Books 3 and 5,). 

In Table 11 can be studied amount learned according to whether or 
not English was the first language of the student. The composite G 
over books for those who had English as their lirst language was* .30, 
wjiile for the other students the composite G was .18.' With the 

exception of Book 3, this variable does not differentiate the 
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students on a learning criterion. Neither do the learning results for 
each book presented for £he various ethnic groups in Table 12 indicate 
any, substantial difference in .composite G scores among the various 
groups. . . - ^ 

Placement Test . The placement examina'tion available for 
Programmed Reading for Adults was^ as speod*fied under "Method"^ also 
included in the pretest/jbsttest battery. The function of the place- 
ment examination^ per the publisher,^ is to start the student at the 
level appropriate for him in the program. On,e student^ for instance^, 
mi'ght need to start^ at Book 3^ whereas a more advanced student would 
start higher in the program^ perhaps at Book 6. 

The learning results are summarized in a' matrix in Table 13. 
There it should be noted that the book number in whijch the student 
started is on the vertical axis of the matrix^ whereas the last book 
completed is on the horizontal axis. It can be observed that one 
student started in Book 3 but went no farther; this student had a 
percentage gain of 25 percent and a G of 1.0- ^ ^ 

Since the placement test is graduated^ the values of G in 
Table 13 were computed on the basis of the maximum score specified 
for the last book in the series completed by .each student^ This, is 
to say that the placement series examiaatioa contains a total Qf 

if 

, 50 itemis. The first 10 of these itasns sample the material through 
Book 3^ the next 10 Items sample through R^ok 4 and so on, Hence^ G 
was computed using 10 points as the possible score for the student 
who studied through Book 3^ and 20 pein^£> f^r those who wt.nt thro'ugh 



Bo 



ok"4.' The possible score through each book is Indicate^ withi n 
parentheses at the top of each cohunn ot J able '13^^ ^ g. ^ the five 
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^•subjects (7h6 started in Book 3 and ended in Book 4»wKo had a gain of 



13 percent a^d the* G value of .57 had that* G value based 




The data'^in Table 13 in the top row show that, in general/ 
thosfe students who started in Book 3 all learned a rathe.r substantial 
amount as$ indicated .by their high G values (an exception is for the 
three students who. ended in Book 7 and had »a G = - .12), e.g., those 
seven who started in Book 3 and ended in Book 8 have a relatively high 
G value of .50. The composite G for these students who started in 
Book 3 is .50. 

Similarly, it can be seen that those ^students who started in 
Book 4 and in Book 5 generally learned a satisfactory amount, as 
indicated by their composite G scores of .50 and .55 respectively. 
In contrast, those students who started in Books 6, 7, and 8 generally 
had depressed G scores. ^ , ^Y**^ y v 

It appears, then, thaKxt-hose s.tudents who had tlie,^Q;9,lL comb'^e.te 
exposure^ to the program learned considerably qioxe per book. Stated 
differently, there is apparently a cumu lative-'ef feet .such that the 
amount of learning for each book is greater/when the series of six 
books have been studied than when fewer, more advanced books have 
,been studied. Note that* -this statement /takes into account the - / 
graduated ceiling on the placement tests. It may^be concluded that, 
within these limits, Programmed Reading for Adults led to a sufficient 
amount of learning, as measured by the Placement Test, providing' the 

students start no later than- Bo^'k 5. '* 

* / 
General Reading Test . Ip Table 14 are the learning data 

obtained from the use of th/ General Reading Test. Again, the 
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results are arranged in a matrix according Co start and completion 
boobs, -of the program. It can be seen V.hic tvo students started in 
Book 3 and completed their studv in Book 3 with a mean G value of .15. 
Seven other students started in Book 3 and ended in Book 4 with 
G = . 18 All students who started in Book 3, regardless of which was 
their last book of study^ had a composite G of .16. A survey of the 
entire matrix shows that, with the exception of cne student'who had 
a G-of 54, the G values resulting from the Gene-ral Reading Test 
measure are, in general, exceedingly low. 

It should be remembered that the General Reading Test was composed 
of items randomly selected from the content tests of all four 
programs. In Table 15 are presented the mean percentages of correct 
items from the content tests for each program as a function of the 
pro^gram studied by the students. For instance, data here are avail- 
able on 110 students who studied Programmed . Reading for Adults . The 
mean score for these students was 92 percent correct on items from 
.-the Programmed Reading for Adults content test, 77 percent from the 
concent test for Bui 1 ding Your L anguage Power, 78 percent for. 
Eeadinig Series and 43 percent from Building Readi ng Power. In' 
essence, then^ measured I'earning for the Programmed Reading 
for Adults wasV greatest by the Prograrr.med Reading for Adults items 
themselves, as would be expected,, A smaller amount; of learning Was 
measured hyj the criteria for Building Your Language Power and Reading 
Series, ^hile the least amount of learning occurred by the measure for 
Building .Reading Power. It is interesting to note tha> approximately 
the same pattern ho-ids for. the other three ijrograms, i.g. ^ regardless 
of which program w^s, studied, the rjighest bccres were obtained on thd 
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Programmed Reading for Adults components of the General Reading Test. 
Similarly, the least amouqr of learning was always measured by the 

■4 , ^ 

-'Sample of items for Building Reading Power. Preliminary data cf these 
kind should be of value in future studies of transfer among programs 
and of the relative difficulty of criierial tescs. ^ 

Adult Basic Lea>rning Examination . Results for the Reading^ 
Spelling^ and Voc'abulary 'scales of the Adult Basic Learning Examination 
were analyzed separately, and che results are presented in Tables 16, 
n J and 18 The various G values , based on the maximum possible score 
for the rests of 58 for Reading, 50 for Vocabulary, and 40 for Spelling 
may all be seen to be extremely lo^?!. This result is not, particularly 
surprising since it is typical to find low learning scores for a pro- 
gram when the measure is )V s tandardized test such- as the Adult Basic 
Learning Examination (cf . , McGuigan, 1967). One obvious reason for 
this is chat items are contained in the test that are not covered in 
the program (and vice versa), so that standardized tests are generally 
not highly valid as criterial measures of prog-rams. This is particular- 
ly true when che -.program does, not purport to include material all che 
way up tc the ceiling of the test. .Hence, the low G scores in Tables 
16, 17, and 18 are quite undets tandabie, thq^ugh ic is apparent that 
whacever is taught by the program is^ noc measured by the Reading, 
Vocabulary, and Spelling scales of the Adult Basic Learning Examination. 

SORL and IRPI . in Table 19 are summarized the results for the 
two individually administered oral Teste (see Appendix D for 
administration and analysis procedures). Nineteen students 
were administered the SORT, their mean pretest level was tl\e 
5.21 grade and their mean posttest level was -5.20 grade. The 
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mean gain was -.01 and the percentage gain was 0 percjent. The 
possible score (the test ceiling) is grlide 10^ so the resulting G is 
0.00.* ■ 

Two mean scores were computed for the IRPI„ The first 'was '^ThjB 
Frustration Level." defined as the number of words in the paragraphs 

^ 

that could be read until the students could not make out any addi- 
tional words. It can be seen that the students on the average could 
read 491.2 words before studying their program and that they could 
read an average of 547. 10 words on the posttesto The mean gain was 
55.9/ and the percentage gain was 11 percent. The possible score was 
778^ the number of words in' the text^ so that the resulting G was 
^ 19..^ The number of errors made in reading up to the frustration level 
was counted^ and a percentage of errors was computed. This error 
value was subtracted from 100 percent^ so that the percentage of words 
correctly read was calculated and entered in Table 19. 
The^ students read a mean of 96.85 percent of the 491.2 words 
correctly on the pretest, and the percentage of correct words on 
the posttest was 96.82 percent. The mean gain was -.03, with a per- 
centage gain of 0 percent. The possible score is, obviously, 100 per- 
cent, a percentage based on the number of words that were *read to 
reach the frustration leve-1. The-resulting G was -.01. Clearly, thes 
measures do not indicate, any noticeable amount of "learning on the part 
of th e stud ents o 

Conclusion for L earning Results 
The placement test indicated that a satisfactory amount of learn- 
ing resulted from the use of this program. * Similarly, the high 
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intelligence students also learned a substantial amount-. In 
contrast^ an insufficient ambunt of learning was indicated by the 
ether measures. 
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LEARNING RESULTS FOR BUILDING READING POWER 
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Learning Results for Building Reading Power 

Content TesXg . This program was used at all sites except #10. 
The results of the content tests for Building Reading Power are sum- 
marieed in Table 20 where^ by surveying the mean G values for each 
book, the general conclusion can be reached that the individuals who 
u:5eu this program did not, on the average, learn a sizable amount, i.e< 
these G values are quite low with none' of them exceeding the standard 
of '.50. Fo^ instance, the 130 students who started in Book 1 had a 
mean pretest score for Book 1 of 80, and a mean posttest score of 89- 
this yields a G value of 144, the highest for any of the books in the 
program. The mean pretest percentage pver the is books, based. on 
1421 scores, was 75 percent. While this is somewhat high prior to 
studying the program, examination of some of ^he values in Table 20 
suggests that it was still possible for G to have shown sufficient 
learning, g. , the mean pretest score for Book 4 was 40 percent and 
the G value was .11. However, as indicated in the "Range" column, *t;he 
mean posttest score- at one site was as high as 76 percent, indicating 
that students with the low pretest score had the good possibili-ty of 
yielding a liigh G value for this book had they but advanced from their 
pretest score of 40 percent to the 76 pertent achieved by others. In 
a similar manner, the highest pretest score was for Book 13, viz., 94 
percent (40 percent was the lowest), the/'gain for Book 13 was 2 percent 
and the G was .31. Hence, sturfent^^ with the highest pretest score h^H 
a noticeably higher G than those with the* lowest pre^^^^scbre (see 
footnote on p. 30) . 
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While there were several relatively high G values for books at 

^/some sites, the*overall evaluation by the content test criterion must * 

be essentially negative*. This conclusion is strengthened by noting 

that' no composite G computed for any site approached .50— -there was 

/ao,'particular site that was consistently high over books for this 

program on this learning criterion. An occasionally high G score 

^t one of many sites i> probably , due to chance fluctuations or to the ' 

effects of other variables not being analyzed. ' ^ 

The relationship between intelligence an^ learning, as measured 

by the- content tests^ may be studied in Table 21. While there were 

^ relatively few students in'tKe high intelligence category, they did 

; yield; pji the average, relatively high G values 

: ; (con:ippsite G = -.56) , noticeably; higher than for the medium and low 

, • intel3.i^ence groups. This suggests .that students who fall into the ' " 
. • ' • »♦ . • " ' ■ . 

; relatively high intelligence category can^l^arn a satisfa9tory amount 



• from t^^, use .of Building Reading Power, .bdj& sthose in the medium and 

^' . • ;;\ . ^ • 

*^»^» ^iow mtel^li^ence categories do not, " 
\;\ \ ^^^^^^;,^\^'' ^ scores for this pnogram did not appear to systemat- 
\\i<iflly vary- w\th' motivation- (Table 2V;.. the^^ all are, furthermore, 

.'A ■ \\A /. ~ 

• 'rfuite. low. ♦ ' ^\ . ; 

X Similarly*, -whether or not English w^s the first language does not 
a^^pe\r-,to be aVefeyant variable for Suilfeng ieading Power since the 
composite G scDres-jwer books >^as,.22 for ..the "English" category, and 

• - • " .'\ ■■ . ' "A 

\« -2^ f(?r.-lhose with Eh^ish as a second language (Table 23). 

n v\ . • 

\ . e is -some variation, in the composite G scores as a function 

• , ^ ot ethpip, group, though -apy general interpretations would necessarily 
^^^^^l^^'^^^i partid.Ljl^arly in view of 'blip' ri^ timely small numbers 

ERIC'.. * '4 , ; ;o . 
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of students for the Spanish-speaking category and for the "other" 
category i^Ta-ble^A) , 

General Reading TestA TabJ?'25;are learning results measured 
by the General Reading Test fo"r students who started in Book 1 and 
ended in various of the 15 books of the program.- The cell of most 
interest is the last, for it contains 61 students who started in 
Book 1 and completed all 15 books of the program. These students had 
a gain percentage of 4 and a G value of .24. Thus the composite G 
values in Table 24 and the composite G for all students of .21 (close 
to that of .24) indicate that by the General Reading Test criterion 
the students did not learnt satisfactory amount from using Building 
Reading Power . 

Adult Basic Learning Examination (Level 2) Results 

The learning -results based on this measure are consistent with • 
those above derived from the other tests, viz., as can be seen in 
Table 26 there are indications here, too, of relatively low amounts of 
leaiiiing. it can \be noted in Table 26 that the composite G values over 
the 15 books for students who started in Book 1 and ended in Book 15 
is .20 on the Reading scale, .10 on the Vocabulary scale, and .12 on 
the Spelling scale, all" relatively low values. ' 

SORT and IRPI 

5 The results on the SORT and the IRPI are summarized for Building 

i - . 

'^^^^^"g Power in Table 2 7. Clearly the results for th^ SORT and the 

percentage of words read correctly on the IRPI , 'do not indicate a 

sufficient .amount of learning. However, the number of words read to 

reach the frustration level on the IRPI was a measure*that did yield 
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a relatively high G score of ,47^ though this -still did not reach the' 
acceptable level of .50. It is particularly interesting to note the 
high pretest level for this high G, 

* 

Conclusion for Learning Results 
The general conclusion based on these various measures ,is that 
an inadequate amount of learning resulted from the use of Building 
Reading Power. 
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PART IV 

LEARNING RESULTS FOR READING SEfilES 
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Learning Results for Reading Series 

r • 

. Content Tests . Reading Series was used at all sites except #10, 

11, and 14, In'^Table 28 are the learning results over sites 

i 

for each of the 20 books of Reading Series . A survey of the G values 
indicates that they are typically rather low, the comt>dsite G over 
books being .31. The highest G value (.49) resulted for Book 1, A 
study of the components^^^of^this G of .49 suggests that while a number 
ofstuSents did seem to learn a satisfactory amount from Book 1 (at 
site #2 o_ne student had a G value of 1.0, at site #3 the mean for 9 
^students was .61, at site ^ the mean of 10 students was ,53, etc,) 
there were considerably more sites with G values below .50. The 
results of the content tests thus indicate that, on the average, the 
students did not learn a substantial amount from the use of Reading 
Series . 

Amount learned as measured by the content tests does not appear 
to systematically i,ncrease with intelligence (Table 29) or motivation 
(Table^30),. This is in contrast to §ome positive relationships with 
regard to these variables previously discussed for other programs. 
Nor do any composite G values in Tables 29 and 30 exceed .5ff; hence, ' 
the program is not sufficiently effective foi: either high* intelligence 
high motivation students. 

In like manner, it does not appear, that the amount of learning 
is substantially different as a function of whether or not English was 
the first language (Table 31), or as- a function of ethnic classifica^ 
tion (Table 32). There is a hint in Table' 32 ^hat Reading Series - 
does not work as well for Spanish speaking individuals as-'for the 
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other ethnic classifications, but one must note the relatively small 
number of students in the "Spa^nisti^' category, particularly after Book if. 

Placement Examination Results . Relatively low G values^are ' \^ 

indicated (Table 33) for the placement test for Reading* Series .- These 
G values were computed on the basis of the maximum score for the last 
book completed^for each student, as specified on- page 44. These maximum 
scores are entered within -parentheses for each book in Ta^ble, 33. For 
example, several ^students started in Book I'and ended iti each book 
through BcTok 12; their G values vary between 0 and .55. The composite 
£ for all these students who starte,d in Book 1, regardless of .where they 
stopped, -was .,36.* • , \ ^ 

The resulfs were somewhat more positive- for the very few students 
who started in Book 2 and ended in each book from 3 through' 9; their 
G values varied between .20 and .85, and their compos ite»; G was .48^ a 
reasonal^y.high value. The composite G scores in the last column of 
Table 33. computed for every other book in which students started never 
exceeded .50. These results are generally corisistent with those from 
the content te^ts in. that they suggest that while some learni,n| did 
occur;, it was, not, by^stablished standards^ a relativ)!^ high amount. - ' 
The. Geneiial Readrnstifc^t- ^^jhs' results for General Reading Test, 
/Summarized in Table 34, indicate that- thd ;g' values are, in genera^, 
relatively low. Twenty 'acid it ional students started in Booli 16 and 
ended in Book 20; they had a percentage gain of 5 percent. and a G of 
.30. The conclusion is t^hus consonant with those based on-the c<^t,ent ^ 
and placeipent tests that, a subs-tantial amount of learning d^d not^,in 
general^ • result from the use of Readitig . Series . 

' ' Adult Basics Learning Examination . The G values are generally 
Ipw for all three scales (Table .35) of the Adult Basic Learning 
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Examination^ a conclusion in line^with the Adult Basic Learning 
•Examination results" for the other programs. In particular, no 
composite G value based on more than ope student ever approached .50. 

SORT and IRPI . The results Jor the SORT and IRPI are summarized 
in Table 36. The G values are all quite Ipjgr indicating an insufficient 
amount of learning by these measures. 

Conclusion for Learning Results 
The general conclus^ion for Reading Series is that, by all 
measures, an insufficient amount of learning resulted from the use of 
the program. 
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LEARNING RESULTS FOR BUILDING- YOUR LANOTAGE POWER 
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Learning Results for Building Your Language Poyei 



Building Your Language Power was used "In fire numbers 1^ 3, 4^ 5^ 
6^ 7^ 8^, and 15. In Table 37 are presented the results for Building 
You£ Language Power as measured b> th.fe c intent t^. it can be seen, •• 
for instance, that 49 people studied Beck 3, that their mean pretest 
score was 86 percent, and that the Lowest; pretest score at any of the 
sitgs was' 77 percent -whiLe the highe-t was 92 percent, 'The mean post- 
test score over all sites was 88 percent, with a range of 80 percent 
to 92 percent. The mean gain from 86 percent to 88 percent can be 7 

seen to be 2, a percentage gain over the mean pretest score of 3 

percent. The G latic was , 19,^v3lae significantly different from 0. 
The range of G valaes 'was .00 to a high of .50 (.50 was "based on but 
three students at one site). ■* 

As before, a set of means is presented at the bottom of Table 37. 
They do convey useful summary infcrmaticn. Based on 157 scores' these 
are somewhat fictitious since they are not xndependent measures (i.e., 
most subject* went through all four books). It can be observed that 
the mean pretest score over si^es aVid over books was 72 percent; the 
mean posttest score was 78 percr=nt, Vieldlng a mean gain of 6 '(8 per- 
cent) ,and an overall G vaL.e of .20. A detailed ^study of the low- 
gain scores, and particolarlv cf the lo^ G scores of Table. 37, indi- 
cafces that a relatively small amoant of learning occurred through 
using Building Your Langu age Power, as measured by the content te'sts. 

1 9 

Wd next turn to the questioo o£ whether or nor this program is more 
effective for students with certain characteristics than for students with 
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other characteristics. Learning .results as a function'6f intelligence * 

J) ~ ^ ' . , 

are presen-ted in Table 38 where it can be noted that most students in 

^this sainple^fell into the lowest intelligence category. L Those classified 

^as'highest in i^ntelligence showed the greatest amount of, learning 

(G.^ -44)^ though they still did not satisfy the standard.of .50, 

In"Table 39 are learning results for students classified according \ 

to whether- they had low^ mediifln or high degrees of motivation. 

For- Book 3^M3 students had a G score of /l3 and a gain percentage , of 

2 percent. =The composite G for tSe low motivation^ students for all ' ♦ 

i 

•four books was .16. Even though it is apparent that the highest G 
score resulted for those classified highly motivated (viz,^ .34) 

even the highly motivate^d students did -hoc evi^dence a .satisfacto.ry ' * * 

' ■ - i» 

amount of learning. , • " ^ 

^ In Table 40 are le^rnilig re^uclts as a. function of ethnic ^ ' 

classification. The students were clasrsified as White, Black, 
^panish-speaking (Puerto Rican. Mexican, Spanish-American) and "other" 
(Oriental and American Indian). The composite G values give a hint 
that Building Your Language Power was relatively more effective for \. ' 
th^ Spanish-speaking and "other" categories^ though none of the 

composiite G scotes are exceptionally high- ' ' - I 

In Table 41 is evidence tKat tudents who had English as their • ^ 
primary language appai;ently learned* less than" did students who had ' , ^ 

another Languajge (almost excluslve(v S.paniish) 'as thei^r first 

language. That is^ English as: a second language^students yielded a ^ 



composite G $core of .40 relative to .14. 
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Table 38, 



'Q Ratios and Percetit Gain on Contend Tests for Builcflag ^ 
Your Language fewer as a Pu\^ction of Intelligence 
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: Table 40, G Ratios and Percent G^in on Content Tests for BGl'lding" ^-^^ ^ 
Your Language Power as a Function of Ethnic Classification 
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Table 41. G Ratios anh Percent Gain on Content Tests for Building 
^ou^ Language fower as- a Function of First Language 
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Ge neral Reading lest Re sults ^ The results for the stufients who 
were pre- and posLtested on the General Reading Test are presented in 
Table >42 'aU students reported cn started in hook 3 and ended in 
Books 3^/ .4, 5. or 6.^ Wi,th the exception of the one student who worked 
only Book 3^ the amount learned as measureo by this test was quite 
small, i.e., the G vaiueb are all low for the individual calls of 
Table 42^ yielding a composite G of .W. ^ 

Results on t\\e Adult Basic Learnin g Examination Level 2 . The 
results on the ^eading^ Vccabalary and Spelling scales for students 
who started in Book 3 and ended in Books '3, 4^ 3. or 6 are presented 
in Table 43* Quite clearly chese G values are all low, indioating 
cha't little learaing took place, according to this measure. 

{Ihe results for the SORT anS IRPI are summarized in Table 44. 
It should be noted that the G values are all quite low, indicating an 
insufficient amount of learning ,by these measures. The general con- 
clusion, consonant with all measures is' t.haL an inadequate amount of 
learning results ifr^i t^h^e use of *:hi5r progi^m. 

' ^^^^ elusion for Le ar ning R esult-s 
All measures *^T^e consistent in' Indicating that-v by a learning 
criterion^ Bu i ld Ing ^ Your Lan guage ^ower is an inef f ective *program. 
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Table 42. G Ratios^ and Percent Gain on Genera]. Reading Test for " 
Building Your, Language Power 



r-^^^e^inni-ag ' Termination Book Number * Composite 

'r Boole Number 3 J i^ T~' 5 6~ G ' 

i ' • \. J' ' ' 
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*P < .05.. 
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PART VI 



INTERCORRELATIONS AMONG VARIABLES 



ERIC 



Intercorrel'^fions' betwear. Dependent^ Variable Scores 




» _ < • ' 

. Before summarizing and interpreting che fmdines, discussed to this 
pointy it may be helpful to. exa.r ine the relationships beween xthe ^ - 
dependent variable^measures. Pearsoji Producti Moment Correlations 
betv^en gach- dependent variable measure for learning and ea.ch other 

taken pa ir-wis.e ' (on the posttest) are presented in Tabl'e ^5. Each . * 

'\ , • . ' 

correlation value was based on the number of students who used both- 

' ' \ c ' ' 

I. as specified within tbe parentheses (hence the varying numbers' 
of students for the different correlations), Fj^ir example,, SUstudents 
took both the Programmed > Reading for Adults content tests and the 
General Read-iog Test; the resulting correlation was ,5%- (p < .05). 
^ Of particular note^ th^ content test for Pi:ogrammed Reading for 

Adults correlated significantly with all possible tests., viz . ^ the /\'^ 
. General Reading -Test , 'the three s^H^^g^f the Adult Basic Learning 
Examination, and the placement test for Programmed Reading for Adults 
(this correlation value is not in the matrix of Table 45>bat^ with 78 r 
students the correlation was .85, p^< 03). A similar fiqding occurs , 
^ for the Programmed Readi ng for Ad alts placement test^ with the excep-- 
•tion that the correlation of ,21 with the Vocabulary scale' of the 
Addlt Basic Learning Examination just misses being significanto 

Surpris ingly^ the Reading Series content tests do not correlate 
significantly with the R eading Series placement test viz,. wit:fe^'^6. 
students the! correlation was only l3. The Reading Series cont;ent 
tests do, tHcugh^ corr^^late significantly anS quite highly with the 
General Reading Test, and also tyith the Vocabulary and Reading scales 
of the Adclt Basic Learning ExaminaTron. The Readin g Series placement 
test, however, does not significantly correlate with any. of the other 
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measures^ making one wonder about i't-s validity. The General 'Reading 
Test very interestingly correlates signif icant^ly ic^th' all scales of 
the Adult Basic Lear^iing Examinatioi)^ viz,, .38^ .69^ -and .49. (Table 
45). In^our rudim&n4:^ry stage of ABE research it is reassuring to 
note that, with the exceptioji of the Reading, Series Placement test, 
there are significant correlations between the other measunes'of 
reading.- ^ ,\ ^ ^ ^ ^ * ' \ ' 
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' P'ART VII 

STUDENT AND PROCTOR EVALUATipNS 
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Student Evaluations ' . ' F'T'^-'j* . 

: > ^ ' t^^ \. ' ■ 

The results from the 7 item Student Evaluation Form for eaah Qf"*V-^>-5c:^V '>^s;^il 
the four pyrograms are summarized in Tables 46 through. 52. Pres^nteifv iT-'r- - ' ' 
are the relative fi^equencies , entered as percentages^ with whiph'<T7' Jl-^"** • ^ 



students checked each point on t;he several scales. For exajnpile;;'^" 



according to the data in Table 46^ 74 percent of Ihe 113 who *s tu4fed . 
Programmed Reading for Adults indicated Lha-t they ''learned mor&" /\ ^ / y^ 

''because programmed instruction was ^sed in this cours e. Tw^ty * 



percent of those 113 students indicated that they believed "it made no 

- difference," while 6 percent were relatively.negative^ indicating that 

they had "learned less" because prograpn'ed instruct ion^ was, used in 

the course. The resalt^ for the, Other thr^''prpgrams can be "Similarly 

studied and compared with aormative data for 33 other programs that 

^ have been empirically t-ested by-McGuilan and Peters (19ff5>. ^. Relative ' . 

to* these normative d'ata^ the' present students thought they learned 

more by using programmed instruction^ regardless of which-^test programs 

they studied, /' • . *' 

/ ' / ' ' 

/ ' . y • ' 

There was a tendency,, slight though it may be,^ :l:or the pregent 

' ' / , '/ ' 

students to believe.. ^hat they learned much more by using programmed' 
instruction than by -Studying regular textbooks ("liable 47n\ In Table 
47 there .is a not^i^e^ble'p'^rcentage who apparerxtiy belpLeve 'they learo 

■ .*'(■■' ' ■ • ■ • •' ■ ■ y 

much more from bc^oks^' as ..indicated by the percentages all exceeding the 

standard 6f 10 p^rcejri^ foy" that column/ How'ever there appears to 'be less 
\ • ' ^ ' / » 

of a preference /for ithose who st(udp.ed the £est programs t6 take another 
V i J ^ ' , r ' '\ % 

course using pro^^kqiiied material than for the norm group (Table 4§) , 
The data in Table 49|^ indicate that those s tudentfi'^who studied > 
Programmed Re'a^i^^ Adults and Reading Series thought they 71^ai^n^d 



I- 





OS 



co- 
ve 



CO 

o 



o 



8^ 
O 



5-2 



8^ 



O 



00 



CO 



ON 
CO 



cn 



1 



CO" 

o 



00 

rH 



•rl 



CO 

a 

•rl 

T> 
CO 
<U 

'60 
a 
'A 



U 




) ■ 



Pl4 



1' 



1 



111 



0)/ 



1 ' - 



00 

c 

•r-l 

c <y - 

H 

• u ^ 

o 

c o 
; itLjo — ' 

t 4-) 

7U 

G M - 

to 3 
0). C50 

- 13 '0) . 

o c 

- >. 

:^ CO' 3 

^ O 
O, 4J M-l 

C G 

- O -r^ 

•M O 
O 3 

• c o 

> 4J M 



0) (0 

(DM W) ^ 

GO CO 

M ,S T-l o 

<D ^ g 'O ^ 

. O O 3 X 



u 

.11 Q ^ <U 

CO ;g jtn f-H 



5-2 

p 



5^ 

in 



5-2 
00 
1-1 



o 



1 T '. v " 



(0 c 
0) 0) 



4 



•r-l 



Q) U U 

C CO -r-l. 

H'-g > 

CO S 



O 



M CO S 



c o 

CO 

0) x: ^' 

>-5 O 4J 



CO 



TJ G 

0)-G. 

i o 

c £ 

03 < 

bO 0) 
o ^ 

CO 




CO 



'.I I 

■ \\ 



.if 

\ 



1 I 



00 



lA 



5-2 
00 



5-2 

in 



1-1 

en 



, / ' ^«/' ^i'.^^r^ err- 



5-2 



CO 



5-2 - 



CO . CO 



00 f-< 

en ON 



5-2 

in 



.1^ 



"A 



U 

o 



U 
0) 

oO 



(0 
0) 



CD 
CO 

50 
C 
•1-1 
XJ 
CO 
0) 



00 
G 
'1-1 

-o 

-H 
.1-1 

3 
P3 



ERIC 



1 ; 2.: 



i 




112 



(A 

. \ 



— 

0) 
4J 

o 
< 

O 

4J 

0) 

u 
:5 



c 
























4J 






- 0) 


CO* 






4J CO 


<U 






CO U 








TO d 


, c 






0) 0 








* <u 












> 




4J 
















0 








4J 




0 


0 y-i 






4J 


o 


B 








(Q 




0) 




}-< 






0) V4 


60 






CO cd 


0 






D Oh 


u 








Oh 
































o 
















4J 


0 






O 


:s 






CO 








0) 


H 






pci 










♦\ 








u 






* C 


o 






0) 


0) 




V 










=> 
















CO 


CO 








CO 



























CO 

H 



PC 



u 

4J CO 

O D 



60 

CM . 



iJ CO 











^ CO 




^ r-l 








♦ < . 




M 00 








Oil r-4 .H 
















•f-i CO C 




< < 





^2 , 
CM 



B 
CO 

•a 
S 



CO 
CO 
60 

o 
a, 



CM 




CO 



5<2 
CO 



IS' 



CO 



CO 
CO 



. -co 



c^ as 

CO CO 



00 
cO^ 

• 0) 



CO 

u 

60 
O 



ERLC 



o 





U 




60 






0) 


> 


C 








CO 


•H 






0 


0) 


•o 






a, 


•H 


CO 




o 






0) 






0) 
60 


Se 






60 


CO 




60 




C 


a 


60 


c 


U 


•H 


60 






0) 






•H 


•o 


:j 


r-H 


La 






o 


•H 


CO 




a, 






0) 


• d 























•V'/ 

i -.7 



0) 

c 

u pNx: 

CO O 
0) 0) 3 
> S 



0) 

c 

CO 

►J Cr CO 



0) -u 

4-) -H 
•H PQ 



CO 

C 3 C 

CO 73 O 

(U dJ g 

S < 











(U -U 




C U 


m 


U -H 




CO 








1-3 CO 





CO U 

a> o 
h3 S 



5^ 



CM 

in 



5<2 


5^ 








CO 




O 


m 






. CM 



C7N 
CM 



»H in 

<f CM 



o 



CM 







^5 


00 


vD 






.CM 





CM 



CM 



CM 
CM 







60 


















c 




























60 














0) 




c 




1 


> 


CO 








•H 














o 


(U 


















•H 


' CO 






to 




u 


O' 




U 


















Q) 








CO 








60 


cn 








M 






60 


CO 




60 




B 


t>0 




< 






60 


c 


U 


CO 


o 








60 




•H 




u 










C 








60 


a, 


60 








TJ 


f-l 


o 


O 




o 








CO 


•H 








u 




















PQ 






PQ 





• .„ . n . 114- 

very much" relatt^ve to tyRe nom students, ? 1^ percent of the 

norm students who ^^reviously had studied iXordgrams checked "learned 

# / 

very much" whereas 36 percent and 41 percent checked this category 

for Programmed Reading for Adults, and Reading Series ; the ratings 
.are less positive for Building Yo :x Language Power (18 percent) 
and for Buildltvg- ^ Reading Power (20 percent). Again^ relative fo the 
norm groap^ all^^our programs were evaluated rather positively on an 
enjoyment crit^rlon^ viz . the percen.tage checking "very enjoyable" 
in Table 50 was noticeably higher for all programs than the standard 
of 17 percent, j The relatively fiigh value of 10 percent under "very 
unenjoyable" occurs for Building Your Language Power o ^ ^ 

Programmed Instruction is sometiti^s Considered to be excessively 
repetitious , because repetition is iatentionally built into programs. 
Nevertheless^*^the agile programmer can reduce boredom^ and the amount 
of perceived Repetition is an important criterion. The data in Table 51 
dicate that all programs more or less approximated .percentages for the 
five categories .of the uorm group^ though Building Your Language Pqwer 
had the somewhat high percentage of 14 percent for "much too much" 

repetition. ^ 

'.1 ' ^ ' . ' ~ 

The ratingSof the written evaluations are overwhelmingly posi- 

tive for* all four programs (Table 52). 

ll ' ^ 

. ^ A summary 'of the student reactions is piesented in Table 53. 

/' 

This sunupfary was based on percentage of positive evaluations onNeach 
item relative the norm group. For example^ 68 percent of the norm 
group checked ^^il learned more" on the first item (Table 46), whereas 
all ^our test /programs had higher p_^rcentages for that category 

A 

( viz . , 74 pe^cfent^ 90 percent;, 88 percent and 85 percent); consequentl) 
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Table 52. Mean Student Reaction to- the :}uestion:- '!in Xc^ur Own' " * 
Wo.rds Say What Thought of' the Program. For Example, 
What Did You Like Abo-ut the Program? What Did You Dislike 
\ about It? Any Other Thqughts-i" (Reaction Was Summarize^ 
as Positive or Negative) 





program 




n 


Positive 


Negative 

\ \ X. 




Programmed Reading 
for Adults 






0 j/o 


y^ 

i7/o ^ 8 




Building Your 
Language Power 




31 


87% 


/ 13% 


\ 


Reading Series 




59 - 


90% ■ ^- 


10 /o 




Buildinp Rpfldino" 

Power 

*" ♦ 




97 


88%' ■ 

V. ■ 


'^'12% ' ' . 
















Table 53. Summary" 


of Student Reactions 


V 


• 


Program 


46 


47 ' 


Table Number 
• 48 49* ■ 50* 


51 52* ' 


\ • 


Programmed Reading 
for Adults 

Building Your 
Language Power 


+ 


+ 


+ + - 






Reading Series 


+ 










I ♦ 

Building Reading 
Power 




+ 
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four pluses are entered in the sununary table under Table 46. In 

evaluating these comparisons^ it should be taken into account that 

only a minority of the 'comparison group consisted of adults^ and 

those were typically literate. Consequently these comparisons are 

tenuous^ ^and again point up the need far basic normative data for 

ABE research. Within such limitations^ though,, and placing heavy 

emphasis on the three most infpo^tant items (those indicated by . ' 

asterisks) ^ , it appears that 'tfhe stiidents who used , all. four programs 

generally evaluated th,em on the positive side, relative to the norma- 

« 

tive data. 

The results of the proctor evaluations of the programs are 
summarized in Tables 54 through 60. We can see in Table. 54^ for 
instance^ th^t of 13 proctors who used Programmed Reading for Adults , 
11 thought tl;iat'the subject matter was academically sound^ while two * 
were^undecided.. It is thus apparent that all programs, were considered 
generally sound by the proctors and that they generally, considered the 
level of the subject matter appropriate for their subjects^ though 
there is somewhat of a tendency for a few to regard each program as 
being too easy (Table 55) . 

£W effective did they consider the programs? Programmed Reading 
for Adults, was considered quite effective or equally effective with 
other^materials by the large majority of the proctors who worked with 
that program (Table 56). Reading Series was similarly evaluated in 
a positive manner. None of the sixteen proctors* who used Building 
Reading Power considered it a great deal more ef^ctive than most 
other materials^ though^four considered it a^Jittle more effective. 
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There is somewhat of a tendency to regard Building Reading Power 
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Table 54. Proctor Reaction by 
" ' Subject Matter of th 


Program to the, Question: "Is 
e Program Academically ^ound? 


the 


Program 


N 


Yes 


No 


Undecided 


Programiped Reading 
for Adults 't 


13 


11 


0 V 


2 


Building Ycur 
Language Power 






1 


0 


Reading Series 


12 ^ 


11 


1 


0 


Building Reading 
Power 




X 


• • 

0 


2 



* i 

Tabl6 55. Proctor Reaction by Program to the Question: "Was the' 

^ Level of the Subject Matter Appropriate for Your Learpers?" 



: ^ 

Program n Too Difficult- Appropriate-- Too/Easy 



Progranimed Reading 
for Adults 


^ — 

13 


1 

1 4 


9 

« 


/ 


Building Your 
Langtlage Power 


• 

7 


■' 1 


5 


/ , 1 

s 


Reading Series 


12 


0 


10 


. 2 


'Building Reading 
Power 


12 


0 


10 ; 


2 
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Table 59. Pr9ctor Reac):ion by Program to the Question: '/^^o Yod^Thinl 
This Program Should Be Available for the Use of Teachei.s'^v 
throughout the Country?" 



Program 
« 

* 


N 




Yes 


No ^ 


Don't Know 


progratnmed Reading 
for Adults 
• 

Building Your 
Language Power 


11 

7 




10 • 

\ 

\ 

\ 1 


\ 0 

\ 

t 
I 

\ 

■ 4 


1 

' 2 


Reading Series 


11 




10 


1 


0 


Building Reading 
Power » 


14 




\ 

\ 

. 8 




5 










. t \ 

y 
\ 




Table 60. Summary of 


Proctor 


Reactions 






Pjrogram 

> 


54* 


' 55 


Table 
56* 


Number 

57 58* 


59** 
^ 


Programmed Reading ^ 
for Adults 


+ 


\ 




+ + 


4- 



B uilding Your - 
^ Language Power 

Reading Series 



Building Reading 
Power 



■J 



.it^i^txyely, with six proctors registering themselves on the negative 
|ide df the scale. Building Your Language Power was even morfe * ^ 
negatively evaluated on the average with 50 percent of t\ie. eight 
proctors regarding it as le'iss effective than other materials and 
50 percent as about as effective. ' ' * ^ jj^ 

With but two exceptions out of 43^ *the proctors were over- 
whefhiingly posiiively inclined to use programmed instruction again 
(Table 57). And, as can be observed in Table 58, those who used 
Prograiraned Reading for Adults ^ Reading Series ^ and Building Reading 
Power enjoyed using the materials with their' students. The obvious 
exception Is with Bui Iding Your Language Power ^ which -was regis^tered 
as "unenjoyable" or "very unenjoyable" by 50 percent of the 8 ' 
proct^j^. Their opinion too was that those three programs stiould 
be maa^ available for the use of teachers throughout the country, 
the clear exception being, again, Building Your ^ Language Power 
(Table 59). ^ . * 

These results for proctor evaluations are summarized .in Table 60. 
It seems apparent that the proctor?, in general, favorablj^ ^evaluated 
Programmed Reading for Adults, Reading $er iefs , and BuildJ.ng Reading 
Power ^ but the general reaction to Build j.ng Your Language Power , ya^s 
negative, ^ , ♦ • * 

, . study Time ; , 

Analysis of the-study log sheets allowed'the computartioji x>f mean 

Study time pe^ 'book ^nd p^ progi^^m. In Ta^tfle 61 can be noted the 
» • . , 

total study time for each program and also t!he mean study ^time for 

' • . , ^ • • 

each beck. • Reading Series , which consists of 20 books, had ^he' longest 
sL-Jdy time. viz . 78.20 hours, an average. Oif 3.91 hours J)er bo'ok. 



Table 61. Average Study Time in Hours per Program Book by Program 



Program 



Average Book 
Time 



Average Program Number 
Time Books ■ 



Progranrni^d Reading 
for ^dujts- 



4.^8 



29.16 6 



Building 'Your 
Language Power 



Reading Series 



5.62 



3.91 



78 Jo" 



- 20 



Building Reading 
Power 



1.06 



15:40 



Error Analysis I '\\ 

One of the standard principles of programming is ffiat error rates 



for the frames should be low (the reason :being to maximize reinfor.Se-* 

meat), ascertaining the error rates for the components *and for the 

entire program^ a judgment of the quality of those components and of 

the entire program can be made. Estimates of the, mean error tates 
« . ' * 

were obtained from a -random sub-sample of programs from the entire' 

sample of 531. The results are presented for the four programs ♦in 

Tables 62 to 65: For example^ according ico the data in Table 62y 

the mean error rate for Book 3 for Programmed Reading for Adults 'Was 

'^5 perf^pnt^ based on six students. The mean error rate for. all "'sdx • 

bcrofc^ based en 20 students was 3.9 percent. A distribution o'f^-'erroV 

rates^ for a variety of programs previously evaluated (McGuigan and 

\ • 

Peters^ '1^965) appears in Figure 2^ allowing judgments to be made^^of ^ 
*^ the values'^^^n Tables 62 through 65 with those of. the distribution. ^=^7^^- 
*^^r instance^ the mean error rate of 3.9 percent for Programmed Reading 

fo^^X^ults appears to be relativply low suggesting that^ by «n ervoT 
^xate \cici>terion^ Programmed Reading for Adufts' ^irs a successful 
prograrn*. urthsrmorc, the mean grror rates are consistently low for 



all booXa:;; \:.. ^\ ^ ^ , ' 



In corit^ast ,\the mean total error rate oT 19.9 pereent for 
Building Yo ui:\ La^ngua ^ Power is excessively high^ as it is for each 
individual book.»)^Table 63x^* According to Table 64^ the mean error 
rate for^ Reading ; )5(feries is 6!^.percent which^ while not excessive^ 

is somewhat hignj^- ^Tji'e problem lieg with particular books^ such 

^ ' ' 

as Book 11 with ah'^tror rate of 9.Kpercent,* Book 7 with ah error 

' 'a' ■ ' 

rate of 10.3 percer{t^- and Books 6 and ^^ith error rates of- 



\ 



127 
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8.9 percent, audls. 1 percent. In any future revision of this program, 
e-^forts should be {made to reduce the error rates of these particular 
books . 



The overall error rate ofs 9,7 percent (Table 65) for Building 
Reading " Power is too high, *a fact that^ is borne out by essentially 
all of the individual books. (Pooks CS 2 and CS 4 are notable excep- 
tions.) 

♦ In' conclusion. Programmed Reading for Adults satisfies the mean 
error rate criterion. Building Reading Pov7er and Building Your Language 
Power do not, and the decision on Reading Series , is borderline. 



Table 62. Mean Error Rate for Progrannned Reading for Adults 

V t 



Bo'ok 



Percent' 
Mean Error 



r 



3 
4 
5 
,6 
7 
8 



Total 



6 
7 
11 
9 
14 
11 

20 



5 
5 

3.9 
3.6 

3.9 



Table 63. Mean Error Rate for Building Your Language Power 



Book 



Percent 
Mean Error 



3 
4 
5 
6 



Total 



10 
10 
7 
7 

10 



19.8 
26.8 
19.7 
10.7 
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Table 64, Mean Error Rate for Reading Series 



Book 



Percent 
Mean Error 



Total 



1 




4 . 






■-^5 


2 




8 






5.6 


3 




4 






. 2.3 ' 


4 • 




4 






■ 5.9 


5 




6 






,8.1 


6 




6 






■ 8.9 


7 




7 






■ io.3 - 


8 




' 7 






6.4 


9 




•5 






•5 .'3 


10 




8 






4.3 


11 • 




5 


* 




9.1 


12 




6 






5.8 


13 




5 






6.7 • 


14 




4 






6.5 


15 




3 ' 






4.8 


16 




4 




# 


5.9 


17 




5 






7.9 


15 




- 4 




5.9 


19 




'3 . 






7.3 


20 


0 


1 . 

17 






• 2.8 
6.6 
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Table 65. Mean Error Rate for Building Reading Power 



A 



Book 



Percent 
Mean Error 



CCl 




20 


CC2 




19 


CC3 




19 


CC4 




19 


CCS 




19 


CC6 




19 


CC7 




18 


CCS 




17 






14 


SA2 




15 


CSl 




16 


CS2 




14 


OS 3 




15 


CS4 




15 


CSS 


• 


14 



14.1 
10.6 
"17.7 
10.0 
9.6 
8.1 
7.5 
12.0-^ 
7-. 8 
8.1 
9.1 
5.7 
8'. 5 
5.0 
12.1 



Total- 



21 




9.7 
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PART VIII 
CONCLUSION 
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CONCLUSIONS, . , 

In Table 66 are summarized the results for all indices for each 
program according to whether the evaluation was positive (indicated 
by +),or negative (indic|t:ed by a -) . The general impression from ^ 
'the learning data^ the primary criterion^ is that none of the programs 
were positively evaluated on all measures. Reading Series and 
Building Your Language Power failed to lead to a satisfactory amount 

learning by all of the indices. There is a suggestion that 
Building Reading Power leads to a satisfactory amount of learning by 
the content test measure for the 'high intelligencer students 'only, 
^imil^ly^ the high int^^IIigence stud^ts on the content tests indi- 
cated a "Satisfactory amount of learning for Programmed Reading for 
Adults. The most positive learning indication is for the placement 
t^st criterion for Programmed Reading for Adults . Within these limits 
Programmed Reading for Adults can evidently lead to an adequate amount 
of learning. 

On the criteria of student evaluations^ proctor evaluations^ and 
error rates^ Pragrammed Reading for Adults may be judged to be a success- ' 
ful program. Building Reading Power was favorably evaluated by both 
students and proctors^ but it had an excessively high mean error rate. 
Similarly^ the stu'^ents and proctors favorably evaluated Reading Series^ 
but portions of this program had error rates that were too high. The 

students favorably evaluated Building Your Language Power, but the 

. , ^ 

proctors did not^ nor did the program have a satisfactory error rate. 
Since Building Your Language Power must be negatively evaluated on all 
criteria except that for student evaluations^ it may be taken as something 
Of a control or 'comparison condition— the suggestion being that the student 
^V^lustion criterion for the Adult Basic Education student is quite uninformative 
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A finals general exclusion from this research is to emphasi2;e 
the need to '<^velop mor^eg^fective reading programs and/or associated 
tests of reading prof iciency"ij The reason for including botl^ the 
learning materials and the mea^sures-^of learning in this statement. is 
that^^. of course^ all our conclusions about learning effectiveness are 
functions of both the programs and their tests. 
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^9^:::::JI^scRiPTioN of programs 




1 
1 

i 



S'everal sources 'of' in'foraation were consMlte<r'5!f^tor^ef*drt^t9>p^'^' * 

more clearly describe the nature of the four reading programs. Tl^e. ' 

analysis of the qb^ervable characteristics^ details such as^the * 

length of the pro-ams in frames, the use of cueing techniques and I:* 

the type^.of res^ses called for were included in the description ' f 

to indicate factd#s which may have had an effect on th-e students* V 
<•«',,• 

performahce, and also to specify more accurately the four treatments 
^(programs). Critical Previews are also included which describe and 
evaluate some of the program characteristics. 

The internal characteristics common to all the programs are 
discussed below. Some of these factors were used in the selection 
procedure and are necessarily common to 'all programs. 

All of the programs consist of a series of booklets used^- in 
sequential order after the student begins working with the material. 
Each has a teacher's manual with a description of the program, with 
each indicating the. program can be used with adults, and suggesting 
supplqjaentary material and exercis-es. ' • . 

The four programs are of the linear-progran^ed variety, offer 
iirinediate feedback as to correct or incorrect responses, and" have IT'"^ 
variety of frame tyf>es^ cueing techniques, and modes of written 
response v^^ithln each. 
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. B. Validation of Programs 

Several sources were consulted tc determine what validation with 

adult learners was available for each program. The Civil Service 

o 

• HI 



cJommis'sidn' Report (1971) lists all four programs and indicates valida- 
tion as not available. Also^ the checklist evaluations by Otto and 
Ford (1966) cover three of the programs used in the fi^ld test. 
Programmed Reading for Adults by McGraw-Hill^ Reading Series by B^L^ ' 
and Building Your Language Power 'by Silver-Burdett. Question 39 
of their checklist^ "materials have bBen f ie Id- tested^ "^S^^arked 
negatively for these three .programs. ' • * 

An effort was made by the Adult Learning Center to obtain data 
from the publishers ' on field testing and revision with Adult subjects 
No data on validation of the programs with adult learners was avail- 
able. 



C. Description of Contents of Each Progranf ^ 

In tly.s section the programs are described as to their component 

^ " ^ . 

parts ^ lengthy type3 of frames and resj^jiyjj^^-^equired^ grade level^ 

1^ ^ - """^^ ■ — 

teaching, me tho(jUand/^ios^ The publisher's descriptions were relied ' 

on for the basic description and orrly those components essential to 
the basic program are described. 

Programmed Reading for Adults ^ authors - Cyntltia Buchanan and 
M. W. Sullivan^ published by McGraw-Hill Book Compan^^ Webster 
Division, New York, New York. 

This program is a seifxes^ -i3f , 8 consumable booklets with about 
700 frames per book. Only Books 3-8 were used in this study--the 
eaflier books require much teacher supervision. * This program has a 
placement test with items seledted from the program. The program is 
designed to go up* to a sixth grade level^ with readability from 1.5 - 
6.0 (Civil Service Commission Report^ 1971). 
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-^l^-eo'rding to the publisher's catalog information, the student 
"will learn spelling, punctuation, grammar, word building, nuances 
Qf meaning, and syllabication." After completing Book 8 (the last 
book^n the program) he .will be able to "read for meaning any material 
written at the fifth grade level." "He will have mastered a vocabulary 
of some 1,500 words^ and be able to genera,liz,e to thousands of other 
words without memorization of phonics rules." The method of teaching 
is described as a "linguistic approach" (1971 Catalog of Instructional 
Materials, 1971). 

^ Building Yo^r Language Power , author - Frank C. Lauback, pro- 

. / 

gr^mmed by William C. Wolfe, Jr., published by Silver Burdett 

Company (a division of General Learning Corporation), Morristown, N. J. 

The Building Your Lan^uaj^e^gower program consists of a series of 
SIX consumable prograramed booklets. For th'is study only Books 3-6 
were used h^pause the books below 3 go to the non-reader level and 
require much teacher involvement. No placemelP4>---tert^ and 
the student simply starts in the first book and works sequentially. 
There are between '300-400 frames per book. The readability level 
established by the Civil Service Commission Report (1971) is from / 
grade 0.0 - 4. 1. v ' ^ * ' ' ■ 

According to the publishers, this pro^|^ teaches "phonetic 

/ 

spellings, 'concise>^ rules, and printed 'and script letters" '(op^ 
letter from publisher by Thomas W. Hendermarh, coordinator of Special 
Education Projects, Silver Burdett Company)* The Civil Service 
Commission Report '(1971) lists the purpose of the program as "beginning 
steps in reading to a vocabulary of 1*300 words." 
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The teaching method described in the Open Letter is through the 
use of f ami liar - illustrations using phonetic jrocedares whenever 
necessary 

Reading Seij-es , author • M. W. Sullivan, published by Behavioral 
Research Laboratories^ Palo Alto^ California 

Reading Series is a series of 20 consumable booklets designed for 
grades 0 to 8 with a readability level from (J.O -.8.0 (Civil Service 
Commission, 1971). A placer^nt test is provided which' incorporates 
selected items from the program Reading Series is the longest program 
of the four tested^ and there are approximately 600-750 frames per 
book. (See Appendix E for a complete list of the booklets and the ' 
number of frames in each.) A necessary component of this 'program .is a 
series of correlated readers which, the student uses after each pro- 
grammed text. ' . . ' 

The program is described as^ 'Linguistically structured to 
present the student with consistent development of sound and word 
Recognition. Logical^ sequential approach with carefully prepared 
illustrations in both the programi^^Kjte>fts and correlated readers 
makes them interesting and appropriate for children, youths^ and 
adults'' (Effective Teaching with Programmed Instruction^ 1970). 

The Civil pervice Coramisl^ion (1971) describes the remediation 
provided in Reading Series as comprehension^ decoding^ and spelling. 

Building Reading Power, authors - Joseph Loretan and Shelly 
UmanSj published by Charles E Merrill Publishing Company, 1300 
Alum Creek Drive, Columbus, Ohio *• 

This program consists of three series of reusable booklets with • 
15 booklets in all. There is n-^ placement test in the program and the 

ii4 
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student sijnply starts in the first book of a series and works 
progressively. The booklets are relaHvely short with approximately 
35-50 frames per booklet (see Appendix E for a complet'fe list of the 
number of frames per book). The pro-am' is designed for grade 4.5 or 
higher (Teacher^s Manual, 1964, p. 2) and has a rea'dability level 
from 5.0 - 7.0 grade (Civil Service Commission, 1971)*. 

The publishers describe their program as one which "provides a 
conceatrated remedial reading program, that is self-administering, 
self-correccing, and self- regulating" (1971 Catalog, Preschool through 
Grade 12, 197 1) . The Civil Service Comntission (1971) lists specific ' 
areas of remediation as word ana^l^^s, vocabulary improvement, and 
comprehension. 

D. Cueing Techniques 

The usual procedure is the use of a variety of cueing techniques 
in each program. It is difficult to, separate cues associated with 
each frame from the rest of the immediately preceding program for 

r 

often even if the student is using the program properly and covering 
all new answers as he goes, there are still cues and correct responses 
which he can use from material which has gone before on each page. 

The cueing techniques identified in the programs are listed 
below and ^re typically used in combination. 

(1) Context clues 

A. Textual, in which some partial meaning is generally 
transmitted through the use of a' sentence .or brief . 
paragraph. 

B. Illustrations, "where a meaning is transmitted tnrough the 
use of a ''cartoon*' or ''picture frame." 
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(2) Partial desired responses where a single letter or group of 
letters blank for each letter and sometime with a separate 
blank fpr each word. 

E. Modes and Frequency of Responses 

Three modes of response frames were identified as applicable to 
the programs tested: selection^ recall^ and practice. A selectio:a ^ 
type response (multiple choice) is' identified as one in which the 
student is required to write a word or letter from memory^ where the 
correct response is not Contained in the frame under study. 

A practical frame is defined as one in which the stuflent must 
merely copy a word or group of .^letters. ^There is no requirement other 
than that the student practice writing the response. ^A fourth type 
of frame is identified which requires no response and is labeled as 
an information frame. 

To obtain ^ better idea of the use of programming techniques 
which place response « demands dn the student^ an analysis was done to 
detei:mine^the approximate percent of each fraiAe type £dr each book 
in the four programs other than the information^ no response^ frames. 
The selection,, recall, and practice frames wer^ -counted for each book, 
in ea,ch program and the peVc^ntage of each was calculated. Table A. 1 
shows a summary of percentage of response frame -types for each program 
Tables E. 1^ E^2^ E. 3^ and E.4 in Appendix* E show the percentage of 
frame type for each book used in each "program. 

As may be observed from Table A.l^ the percentage of type respons 

/■ ' ^ / ^ 

called for were primarij.y in the selecti6n ^and recall categories for 

all programs other than Building Your Language Power . " Bui^lding Your 

Language Power utilized extensively practice fr^me. 
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Table A, 1, Summary Table of Response Frames by Type 







Percent Frame T>'pe 




Program 


Selection 


0 Recall 


Practice 


Programmed Reading 
for Adults 


64 


36 


0. / ' ' 


Building Your 
Language Power 


17 


24 


59 


Reading Series 


44 


56 


0 


Building Reading 
Power 

^ 


51 


49 


0 
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APPENDIX B 
PROCEDURES FOR PROCTORS 
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sjep-Sy-step role of proctor 



1. Conduct the initial inter^/iew (Fcrm 1). During this time and 
throaghout the testing be alert to eye problems of 'the student. 

2. Explain the potential value of the course to the student an^i 
demonstrate what programmed instruction is and how it works, 
cover slides. 

3. If this^ is a new student^ familiar-isr^him with the facility and 
its purpose, other activities ^ing on^ restrooms, available ^ 
refreshments, etc. ^ : . 

W ' ^ 

4. Administer ABI,E Level II Form A - Vocabulary, Reading and Spelling 
only. If ABLE raw score (number right) in reading is between 23 
and 49 inclusive, the student will be in the study. Should the 
student score, below or above these levels, he will not be in the 
validation study and can be terminated at this 'time. 

5. Begin a file folder for each student. 

^6. Assign a student number and determine the program he will be in 
' by using the Student Roster (Form 6). If on the Student Roster 
the student is. to be placed in a program that falls outside the 
range of his ABLE raw score (number right) in reading , move down 
the list to the next available program in which' his score falls. 
Skipped programs should be filled by the . f irs't available student 
who scores yithin the -ABLE range. 

7. Administer the General Reading Test, * ' 

8. Administer placement test, if any, for the particular program in 
which the student is to work and place in the program accordingly 
- administer the pretest for the book in^which'he is to .begin s * 
Refer to the Master Check Sheet for each individual program. ' 

9. Instruct the student in the use of the Student Time Card (Form 2) 
. and follow through each visit to be sure he anderstands and fills 

it out correctly. ^ . . 

10.. After^the siHident has begun his progtiam, administer Standard 
Progressive Matrices (Raven 's)i«» 



LI. 



At the completion of the program or wh^n this student must leave 
the project, and ^e has completed 4t least one book, administer 
tfie posttests for the particular program in which the student 
worked. Refer to the Master Check Sheet. 



Date 

Pre Post 
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12. Ask the student to fill out the Student Evaluation Sheet (Forjn 4) • 
Encourage him to be frank. ' Explain that he is doing this Anony- 
mously. You may help the student understand this form by 'explain- 
ing^ defining words^ etc. 

13. Re-administer the General Reading Test. 

14. Re-administer ABLE Level II Form A. 

15. Share with- the student results of his growth as shown by any of 
* the posttests. ^ ' - 

16. When the student has finished the program and all the fprms are 
complete^ please place them in a -secure file for collection by 
an Adult Learning Center representative. 



■ \ 
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SUMMARY SHEET OF GENERAL INSTRUCTIONS FOR PROCTORS 



1. Kindly famij^arize yourself thoroughly with all tests and programs 
and procedural farms involved in the study. 

^. During the time the student works in this validation program^ it 
is requested that he work in no other reading program or course. 
If it is necessary for his progress that he take another course^ 
'it should be one not specifically designed to teach reading - 
math preferred. 



3-. 



5. 



- Please make the student aware that we arp helping hiih^ and he'^in 
'turn is helpin^^is. 

Throughout the testing and program study^ guide the student in , 
procedural matters, but offer NO help where content is concerned^ 

Regularly keep track of each student's progress on the Master 
•Check Sheet for Proctors. Check each item as completed. , There 
, is a different Master Check Sheet for each program, ' ' 



^' ^ UA4e r no circumstances should students allowed to remove . 

^^{/rogrammed materials from the room Where instruction takes place, 

A careful time check must bexkept on the use of the material - 
, ' use the Student Time Card. (Form 2), 

Kindly specify any course now being studied by the stu'dent in 
addition to the reading program used fqr thts project/ Use the. 
Student Course Record (Form 3) to supply this information. 

8. Jit the completion "of the program fill out ^ Proctoy Evaluation 

Sheet (FQrm 5) for each of the reading .programs, with'which you 
worTced directly. P 

• • % 4 

9, -^It is most importa'nt that all forms^ workbooks and tests t,liuw * 

* clearly the student name -and testing site, 

10. Please do not go over any^of the' pretests with the student, , 
because it will affect the student's score on the posttests. 
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APPENDIX C , , ' 

RELIABILITY ESTIMATES OF ADULT LEARNING CENTER 
CONSTRUCTED TESTS 



' . - . ; 161 ' , 

Table C. 1. Spearman-Brown *Split ^Half Estimate of Reliability on ALC 
Constructed Test's* ' ' 



P 



Number Items ^ 



C6A^^ SA Book 2 
/.vj\Cont^t>^CS Book 1 
\^ Content/ \eS Book 2 
' Content, cS/Sook 3 
V Content, CS Book 4 
Content, CS Book 5> 



General Reading ^ 't 

Building Your " Langua^ge Power 

7 Content, Book '3 - . \^ 

• • * 

Content, Book 4 * 
Content, Book 5 • T-^ 

^ Content, Book 6 
Building Reading Power 
J Content, CC3ib.Qit.l . 
Content, CC.-figc^-^*^^^ ' 
Content, CC B^j^k^' 
Content, CC Sook-V ^ ' 
' ^Content, CC Book 5 
;^nt, CC Book 6 
:^;teQntertt\ CC Book 7 
\ . l^c^^tent, CC Book 8 
SA Book 1 



888 
757 
601 
795 
'333 
198 
806 
260 
780 

: 610 
.850., 
.581 
.930 
.930 
. 787 
.§16 
. 773 
.148 
. 750 
.437 
.449 



4? 

9% 
t 

•23 

* . 

17 
21 

155 

: 5 

u - -•• 
11 

12 
13 
10 
15 
12 
15 

i5 

5 

5 

13 
8 
5 



•4 •', 



f4 



50' 
.38 

< 

37 

36 

X5 

2? 

42 

39 

31 

34 

23 

29 

30 

29 

28 

24 • 

24 . 

24 • 

23 

24 ^ 
l' 

(23: 
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APPENDIX D 



PROCEDURE FOR ADMINISTRATION AND ANALYSIS OF 
THE SLOSSON ORAL READING TEST AND 
INDIVIDUAL READING PLACEMENT INVENTORY 
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Administration ' 

V 

The procedure for administering the SORT "^required that the % 
student read as far as possible through 9 lists of words with 20 
words per .list graded in difficultyo After being informed that the 
tape recorder was running, the stutient began reading the words 
consecutively^ stopping when he reached the list where he could not 
read any of the words. , 

The procedure for the administration of the IRPI was much the 
same„ The student began reading the six paragraph cards and continued 
reading until he could not make out any of the words. 

For all recordings the student and proctor sat together^-the 
proctor placed the SORT word list sheet and the IRPI paragraph cards 
one by one in front of the student. The same procedure was used for 
both pre- and posttesting. * The SORT and IRPI were administered in ' 
the same session, , 

Analysis 

The magnetic tapes were se^nt to Dr. Edwin H. Smith of Tlorida 
State Univers-ity for analysiSo The standard analysis procedures for 
the SORT are simple tabulations as follows: 

The total number of correct words pronounced were counted in all 
of the lists and this total divided by two equals the grade level. 
For example. If a student vac able to read 164 wcrds correctly out of 
200 words on the test^ his grade level would be 164 4- 2 =^ 82 or 8.2 
grade^ level^ 8th year ^ 2nd month. 

The mean grade levels for the students in each program were 
calculated for both the pre- and posttests, gain, percentage 
gain and G. Maximum Possible Score ^ 10 >0 grade. 



ERIC 



The analysis for the IRPI is divided intd'two parts. First, on 
both pre- and posttests the total number of words read up to and not 
including the point where the student makes 10 oral errors on one card j[ 
were counted. The 10 error point is called "the frustration level," 
and the e^lpected effect would be that a good reading program shoulc^ 
move the student's frustration level up, _i.e. ^ he would read more words 
before he made the 10 errors. The results are shown in the tables for 
the students in the four programs, in the rows labeled IRPI, * 

Oral errors were also counted up to the frustration, level under 
the following 14 different categories: 

repetitions ^ j , 




1^ 
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The pxal errors in all categories were summed for each student 
for both ^he pre- and posttest. A decrease' in the mean number of oral 
errors would be expected if the programs increase overt reading per-^ 
fomance. 



( 
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APPENDIX E , 

PERCENTAGE OF RESPONSE TYPES FOR THE. BOOKS 
USED IN EACH OF THE FOl^R -PROGRAMMED PACKAGES 



Table £• 1: 'Response-Frame Types by Book for ^Programmed Reading for 
Adults 



Book • 


Selection 


Peroent Frame Type 
Recall 


Practice 


Cases 
Sample 


3^ 


88 


12 • 


•0 


/' ■ 753 


• 4' 


90 . 


• 10 


0 


706 


5 


70 


30- 


0 


663 


' 6 


52 


■ . 48 


0 


819 


7 


' 44 


56 


0 


878 


8 


is - . 


62 


0 


824 


TOTAL 


" 64 


36 


0. 


• 4643 



Tal?le E..2. Response-Frame Types by Book ^ for Buildj^g Your Language Power 



Book 


Selection 


Percent Frame Type 
Recall 


P;ra'ctice 


Cases 
, Sampled 


3 


25 


22 


53 


426 


4 


12 


24 


' 64 


244 


■ 5 


17 


26 


57,, 


267 


6 




24 


63 


244 


TOTAL . 


17 . 


,24 


\59 


1181 
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Table E, 3, Responses-Frame Types by Book for Reading Series 







Percent Frame. Type 
1 Re^call 


rraciiice 


Cases 
Sample 


1 




53 


0 


672 


0 


*+ / 




0 




3 


; A3 


57 


0 


598 


4 


34 


66 


0 ■ * 


634 


5 


^ * . 2 2 


78 


0 


610 


6 


28 




0 ^; 














- 1 


39 


61 ! ■ 


0 ' ! 


' 672 


8 


43 


■57 




704 


9 , 


30 


70 




706 


in 


JO 


6/ 


0 


"7 0 0 

722 


11 


45 


55 


0 - 


728 


12 




- 41 1 


0 


734 


13 • 


69 


31 


0 


736 


14 


61 


39 


0 


736 






r 






15 


43 


.57 


0 


73& 


16 


44 


56 ' 


0 


7.36 


17 






0 


7 '^A 
/ JD 


18 


45 


55 


0 


736 


19 


48 


52 


0 


736 


20 


51 


49 


. 0 


736 


TOTAL 


44 •" 


56 


0 


13,100 



ERIC 



, \ i U J . 



173 

Table E. 4. Response-Frame Types by Book for Building Reading Power 



Book 



Percent Frame Type 
Selection ^ Recall , Practice 



Cases 
Sajnpl^ed 



CC-1 




39 




41 






CC-2 ' 




43 








O "7 

37 


CC-3* 




46 








• 

,35 


CC-4 




68 


• 




2 


♦ 41 


cc-5 




57 








42 


CC-6 




44 






o 
2 


54 


CC-7 




*+ / 








45 


CC-8 




71 








41 


SA-1 








/u 




43 


SA-2 


» ^ ' 


44 




56 




36 


L»Or" A. 




00 




32 ' . 




37 


CS-2 




^ 48 








42 


CS-3 




65 




35 




43 


CS-4 




43 




57 




■42 


CS-5 




32 




68 




37 


TOTAL 




51 




49 




612 














4 



^ ■ - 
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